Fetch URL Content

[Anon]

I have the following 2 scripts based upon user input from this board. The scripts basically fetch the contents of a remote site (URL). The scripts work fine for a domain (www.phpbuilder.com) but the control hangs when used with a domain link (www.phpbuilder.com/forum) or a URL which does not exist. Any clues why is this happening?

<?
$url = "www.phpbuilder.com/forum";
echo strip_tags(join('',file($url)));
?>

AND

<?
$url = "www.phpbuilder.com/forum";
$fp = fsockopen ($url, 80, &$errno, &$errstr, 30);
if (!$fp) {
echo "$errstr ($errno)<br>\n";
} else {
fputs ($fp, "GET / HTTP/1.0\r\n\r\n");
while (!feof($fp)) {
echo fgets ($fp,128);
}
fclose ($fp);
}
?>

[Anon]

You have to specify that this is a http request.

$url = "www.phpbuilder.com/forum";

should be

$url = "http://www.phpbuilder.com/forum";

-jens

[Anon]

You can NOT open a socket connection to a URL. You open a socket connection to a valid domain name. Your example is hanging because your DNS system is trying to look up nonsense.

To fetch a specific URI from the a valid hostname, the GET command should be modified to include the URI:

GET /forum \r\n\r\n

However, there are two more potential problems:

• This will NOT handle a redirect, if one is returned by the remote server. You would need to read the response headers, examine them, and act accordingly. (This also is a problem with the built-in HTTP support in fopen, et cetera.)

• This will NOT work with name-based virtual servers. You would need to modify the HTTP request to conform with HTTP 1.1 requirements. Basically that involves changing 1.0 to 1.1 in the GET line and following it with a "Host: www.whatever.com" line.

[Anon]

Thanks for the information. Could you please suggest a fool-proof method to fetch a URL's contents. Any help would be great!

[Anon]

Well, nothing is foolproof. If you fetch an invalid URL, you're going to have problems.

When I want to pull in data from a (valid) URL, I simply use file(), which returns an array, and then implode() the results to put it into a single string. The only time I have needed to do socket-level connections is when I want to examine HTTP headers or use some other protocol, such as WHOIS.

[Anon]

Simply put, what I want to do is to get some sort of an error message or an empty value when an invalid URL is specified to the file() or fopen() functions. But the problem is that the script goes for a hang when an invalid URL is given, and I can't do too much because of this irregularity.

BTW, any way to timeout a script (by code).

[Anon]

Try the gethostbyname() function on an invalid domain name. If the function hangs (or has a long timeout), it's your DNS server which has difficulties to handle invalid names. And you won't be able to do much to avoid the problem...

[Anon]

You can always suppress error messages from functions, by prefixing them with @

Ex.

<?
$url = "http://www.phpbuilder.com/forum";
echo strip_tags(join('',@file($url)));
?>

Should work.

🙂
jens

ASHBeN wrote:

Simply put, what I want to do is to get some sort of an error message or an empty value when an invalid URL is specified to the file() or fopen() functions. But the problem is that the script goes for a hang when an invalid URL is given, and I can't do too much because of this irregularity.

BTW, any way to timeout a script (by code).