Obtaining Source On Remote Site...

Krakken

Hi, I am pretty sure this is possible. I simply want to wiew the HTML source of a remote website for parsing within the PHP file. How can I go about this?

bj_

I am pretty sure you have to use sockets to connect to the remote computer and grab the html file.

Here is the article on sockets
http://www.phpbuilder.com/columns/armel20010427.php3

TruckStuff

Open it like a file:

$url = 'http://www.yahoo.com';
$in = fopen ($url, 'r');
$source = fread ($in, 1024);
fclose ($in);

Krakken

Okay I am using the following code but for some reason it only returns the first 35-40 lines of the website. I noticed on both pages I tried it on (same website) that there are about 7 carriage returns in the code where it stops reading.

This is the code I am using:

<?
  $webFile = 'http://myplanetside.station.sony.com/character.jsp?charId=325513&worldId=21';
  $filData = file_get_contents($webFile);

  echo htmlspecialchars($filData); 
?>

The output I get is:
http://www.sapian.co.uk/other/test.htm

Can anyone diagnose why it only reads a few lines then stops?

HalfaBee

They must check for the browser there and stop if you are not one.

HalfaBee

Krakken

Nope I asked their support team. There is no such protection in place. Maybe I could try another way to read the source? Maybe it is timing out?

HalfaBee

For some reason they send a NULL ( 0x00 ) at that point which seems to be stopping the php functions from reading any further.

I tried reading 1 byte at a time and found this result.

< 26
t 74
d 64

26

a
0

This is where it ends and it will go no further.
Maybe they will fix the problem for you. 🙂

HalfaBee

Krakken

Strange 🙂

Is there any way to "manually" bypass this? I seriously doubt they would change their source just for me. Maybe even a different way to do what I want. I am simply trying to get values from the website and I thought scanning the source would be best.

Thanks.

ucbones

Hey,

as I understand it,

$url = 'http://www.yahoo.com'; 
$in = fopen ($url, 'r'); 
$source = fread ($in, 1024); 
fclose ($in);

will only load the first 1024 bytes of the code into $source, try upping that number. I might have got the wrong end of the stick though...

ucbones

Krakken

It makes no difference to the problem I have.

kevinsequeira

<?php
$file = fopen ("http://myplanetside.station.sony.com/character.jsp?charId=325513&worldId=21", "rb");
if (!$file) {
    echo "<p>Unable to open remote file.\n";
    exit;
}
while (!feof ($file)) {
    $line = fgets ($file, 1024);
			echo $line;

}
fclose($file);
?>

reg
kevin

Krakken

Exactly the same result :/

kevinsequeira

well i just tested it out and it showed up the whole site !!!

reg
kevin

Krakken

Got a link?
http://www.sapian.co.uk/other/test.htm

Krakken

Hmm this is very strange. It seems to work fine on my other host. It must be a host problem. :/

Would there be any setting that would affect this? I have full access to edit it.

kevinsequeira

what is the OS on both the hosts ?? what versions of PHP are u using

reg
kevin

Krakken

Linux on both and the one that doesn't work uses 4.3.1. (Not sure about the one it works on but older I think)

kevinsequeira

and this is all the code you have on the page

<?php
$file = fopen ("http://myplanetside.station.sony.com/character.jsp?charId=325513&worldId=21", "rb");
if (!$file) {
    echo "<p>Unable to open remote file.\n";
    exit;
}
while (!feof ($file)) {
    $line = fgets ($file, 1024);
            echo $line;

}
fclose($file);
?>

reg
kevin

Krakken

Yes

HalfaBee

It didn't work on my host.
I tried all the variations above.
What version of php are you all running.
I'm on 4.3.1 FreeBSD

[edit] I missed Krakens post about 4.3.1

HalfaBee