retrieving search engine results

johncook

I'm trying to retrieve search engine results using PHP (so I can process the results myself) and something weird is going on. For starters as a test, I wanted to just call the results and output them to see if it was working okay. It always cuts off halfway through the page. Any ideas on why it might do this? It's definitely not a time-out thing cause it happens instantaneously. Here's the code:

<?php
$url = "http://www.google.com/search?sourceid=navclient&ie=UTF-8&oe=UTF-8&q=test";
$fp = @fopen($url,"r"); 
$content = fread($fp,100000);
echo $content;
?>

bernouli

you may use [man]fsockopen[/man]

$url = "www.google.com";                

$fp = fsockopen($url,80,$errno,$errstr,30);                                    

if (!$fp) {
       echo "$errstr ($errno)<br />\n";
} else {                                                 

    $out = "GET /search?sourceid=navclient&ie=UTF-8&oe=UTF-8&q=test HTTP/1.1\r\n";                         

    $out .= "Host: [url]www.google.com\r\n[/url]";                                            

    $out .= "Connection: Close\r\n\r\n";                                           

    fputs($fp, $out);                                

    while (!feof($fp)) { 
        echo fgets($fp, 128); 
    } 
    fclose($fp); 
}

johncook

Thanx for the reply, works much better. Simple outputing puts all this jargon at the top but no matter, I don't want to redisplay the page, just parse the results so for my purposes, it works great. Thanx! BTW, the line in $out about Host: caused a problem with Google so I removed it - this was my final code.

<?php
$url = "www.google.com";                 

$fp = fsockopen($url,80,$errno,$errstr,30);                                     

if (!$fp)
	{ 
    echo "$errstr ($errno)<br />\n"; 
	}
else
	{                                                  

    $out = "GET /search?sourceid=navclient&ie=UTF-8&oe=UTF-8&q=test HTTP/1.1\r\n";                          

	$out .= "Connection: Close\r\n\r\n";                                            

    fputs($fp, $out);                                 

    while (!feof($fp))
		{  

        echo fgets($fp, 128);  

    	}  

	}
fclose($fp);  

?>