Getting info from other sites

drndrw

Hey guys. I am currently trying to write a small script to find a site's page rank. But to do this, I need to get information from another site. I know how to send it, but not receive it. Is there a function to do this? Thanks.

maxpup979

It kind of depends on the site, and how you are extracing the info--If you are just grabbing a page, you can use fopen("http://www.example.com/", "r").

Hope this helps

NogDog

If you want the HTML text of an entire page in one variable:

$html = file_get_contents('http://foo.bar/page');

drndrw

Ah okay. Well actually, I really just want to get one or two numbers. Is there anyway to do that?

NogDog

Without any more details on what you get back from the "other site", there's not much we can tell you. If it's a web page that you want to "scrape" for information, then you'll need to grab the whole page, such as I suggested with file_get_contents(). Then the string that is contained in the variable to which you assigned the results of that file_get_contents() call can be searched via any of the applicable PHP string functions to get the data you want, assuming there is some consistent pattern of text or mark-up that can be used as a basis of that search.

If you are talking about some other sort of interface, such as calling a script on another site which returns HTTP post data, or an XML file, or some other sort of data, then we'll need to know more about that interface before we can suggest anything.

Scriptmaker

Also, if you are looking to just use someone else's script/site to allow visitors to your site to check their pagerank, I'd suggest getting some better practice by learning how to access google's api and write your own pr script.

drndrw

Ah, I didn't even think about Google's API. Maybe I should look into that.

The-Master

You can also use cURL to send post data, and receive page contents. Curl is also faster that file().

NogDog

The-Master wrote:
You can also use cURL to send post data, and receive page contents. Curl is also faster that file().

While I'll concede the possibility that it might be true that a cURL request is faster that file_get_contents(), do you have any empirical proof of this, or a reference to any documentation that does? (No offense intended here, it's just that I've seen enough examples of speed difference claims that did not stand up to empirical testing that I tend to be a bit skeptical of any such undocumented statements.)

The-Master

It is meant to be faster than fopen and fread. I thought it was file() that it was faster than. I don't know if it is, I would run some tests but my server is down.

http://curl.haxx.se/libcurl/php/ wrote:
Performance Comparison*

10 per minute for fopen/fread for 100 HTTP files
2000 per minute for PHP/CURL for 2000 HTTP files

NogDog

I tried this little test:

<?php
ini_set('display_errors', 1);
error_reporting(E_ALL);
ini_set('max_execution_time', 240);
function doCurl($url)
{
   $ch = curl_init();
   curl_setopt($ch, CURLOPT_URL, $url);
   curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
   $store = curl_exec ($ch);
   $html = curl_exec ($ch);
   curl_close ($ch);
   return($html);
}
function doFgc($url)
{
   return(file_get_contents($url));
}
microtime(true); // get timer in cache
$runTimes = array();
$url = "http://www.charles-reace.com/";
$start = 0;
$end = 0;
$runs = 10;
$startix = rand(0,1);
$runs += $startix;
for($i=$startix; $i<=$runs; $i++)
{
   if($i % 2)  // alternate sequence methods used
   {
      // cURL:
      $start = microtime(TRUE);
      $html = doCurl($url);
      $end = microtime(TRUE);
      unset($html);
      $runTimes['curl'][] = $end - $start;
      // file_get_contents:
      $start = microtime(TRUE);
      $html = doFgc($url);
      $end = microtime(TRUE);
      unset($html);
      $runTimes['fgc'][] = $end - $start;
   }
   else
   {
      // file_get_contents:
      $start = microtime(TRUE);
      $html = doFgc($url);
      $end = microtime(TRUE);
      unset($html);
      $runTimes['fgc'][] = $end - $start;
      // cURL:
      $start = microtime(TRUE);
      $html = doCurl($url);
      $end = microtime(TRUE);
      unset($html);
      $runTimes['curl'][] = $end - $start;   

   }
}
printf("<p>Total time for cURL: %.6f sec, average: %.6f sec.</p>\n",
       array_sum($runTimes['curl']), array_sum($runTimes['curl']) / $runs);
printf("<p>Total time for file_get_contents(): %.6f sec, average: %.6f sec.</p>\n",
       array_sum($runTimes['fgc']), array_sum($runTimes['fgc']) / $runs);

A sampling of results:

Total time for cURL: 10.608718 sec, average: 1.060872 sec.
Total time for file_get_contents(): 7.226971 sec, average: 0.722697 sec.

Total time for cURL: 11.559411 sec, average: 1.050856 sec.
Total time for file_get_contents(): 8.025178 sec, average: 0.729562 sec.

Total time for cURL: 13.719659 sec, average: 1.247242 sec.
Total time for file_get_contents(): 8.441728 sec, average: 0.767430 sec.

Total time for cURL: 11.574428 sec, average: 1.157443 sec.
Total time for file_get_contents(): 7.953396 sec, average: 0.795340 sec.

So at least for the purpose of a one-time call to another site to get the html content of a page, I'll stick with file_get_contents(). (I suspect there may be an advantage to cURL if you are making multiple requests over the same connection, but for a one-off deal file_get_contents() appears to have the edge.)