file_get_contents & Bungie.net

[marshdabeachy]

Hey all, I've got a quandry. If you've ever browsed any Halo 2 forum, you may have seen something I made - the Halo 2 signature. Basically, it's a Flash app that uses PHP to retrieve the HTML from Bungie's website and sends that back into Flash for parsing and displaying stats.

Yesterday, Bungie made a change to their site, and now my PHP script is literally only returning the last line of the HTML. The script can return other websites fine, AND the same problem occurs from other servers, so I know it's nothing on my end. However, I need a way around it, as thousands of people use this signature and and I'm starting to get e-mails.

Here's the script, minus the hit counter:

<?php
extract($_REQUEST);
echo "file=" . urlencode(file_get_contents($filename));
?>

Obviously pretty basic, it just escapes the HTML and returns it to Flash. Here it is in action: Click

Like I said, only the last line. Now, view the source on the actual page and you'll see there's a lot more: Click

I have no idea how Bungie could accidentally break this, but I need to find a way around it. I don't care how, it just needs to grab the entire HTML from Bungie's page.

Ideas?

[drew010]

ive seen this problem on a few sites now. Seems they are checking what browser you are using to determine what css or html to use. you will need to make it a bit more complex, and use some socket functions like [man]fsockopen[/man] so you can pass a user agent like Mozilla/5.0 along or something, or you can use curl, but its simple enough with fsockopen since you dont need any cookie handling

[marshdabeachy]

You'll have to excuse me, I'm more intermediate at PHP, not an expert.

I found a function in the comments of the fsocketopen link you provided, but it's still returning the same thing:

<?php
function fetchURL( $url ) {
   $url_parsed = parse_url($url);
   $host = $url_parsed["host"];
   $port = $url_parsed["port"];
   if ($port==0)
       $port = 80;
   $path = $url_parsed["path"];
   if ($url_parsed["query"] != "")
       $path .= "?".$url_parsed["query"];

   $out = "POST $path HTTP/1.1\r\nHost: $host\r\nContent-type: application/x-www-form-urlencoded\r\n${others}User-Agent: Mozilla 4.0\r\n";

   $fp = fsockopen($host, $port, $errno, $errstr, 30);

   fwrite($fp, $out);
   $body = false;
   while (!feof($fp)) {
       $s = fgets($fp, 1024);
       if ( $body )
           $in .= $s;
       if ( $s == "\r\n" )
           $body = true;
   }

   fclose($fp);
   return $in;
}
$str = fetchURL("http://www.bungie.net/Stats/PlayerStats.aspx?player=dilinator");

echo "test=" . $str;
?>

I'm not too familiar with the path headers, did I screw something up there?

[drew010]

i managed to get it with this:

<?php

$fp = fsockopen("www.bungie.net", 80, $errno, $errstr, 30); 
 fputs($fp, "GET /Stats/PlayerStats.aspx?player=dilinator HTTP/1.0\r\n");
 fputs($fp, "Host: www.bungie.net\r\n");
 fputs($fp, "User-Agent: Mozilla/5.0\r\n");
 fputs($fp, "Connection: close\r\n\r\n");  

while(!feof($fp)) {
  echo fgets($fp, 255); 
}

?>

[marshdabeachy]

hugs Drew

Thank you! You just made 1000s of people happy! :p

[drew010]

hehe cool glad to have helped such a big project.