Problem with fopen()ing URLs

Kreuzberg

Hi,

I use this code to open remote URLs:

$fp = fopen("http://any-url.net", "r");
$content = "";
while(!feof($fp)) $content .= fread($fp, 8192);
fclose($fp);

This works fine.

But for some reason I get the following error message with certain URLs:

Warning: fopen(http://problematic-url.net): failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found
in ... on line ...

This happens even though the URL definitely does exist, which can be verified with any web browser.

Certain URLs always work, certain URLs always fail.

Just for example, all URLs under www.meinewebseite.net/... (e.g. www.meinewebseite.net/mueller) fail, while www.meinewebseite.net/ itself works.

I wasn't able to find anything that all problematic URLs have in common, so I'm stuck with the problem.

I am using PHP 4.2.2 as an Apache 1.3.26 module under Linux.

Thanks to anyone who can help me!

Philipp

sid

check www.php.net/fsockopen

schoene gruesse nach berlin ;-)

Kreuzberg

Thank you and schöne Grüße aus Berlin, but fsockopen() does not solve the problem.

$socket = fsockopen("problematic-host.net", 80);
fputs($socket, "GET /problematic-path HTTP/1.0\n\n");
$content = "";
while(!feof($socket)) $content .= fread($socket, 8192);
fclose($socket);

gives me a 404 error message in $content, though it works with non-problematic hosts.

Weedpacket

I watched the headers sent out from the server when I requested www.meinewebseite.net/mueller - it sends out a 404 error, containing a frameset. Inside the frameset is a single full-size frame with a src of

http://www.meinewebseite.net/cgi-bin/hpm_show.cgi?userid=65611239&name=ice

which is of course requested. The response to that is a 302 Moved Temporarily, including a Location: header reading

designs/ice/index.cgi?page=index&typ=&id=&userid=65611239&var=&ustatus=

http://www.meinewebseite.net/cgi-bin/designs/ice/index.cgi?page=index&typ=&id=&userid=65611239&var=&ustatus=

is requested, and that, finally, is the page you're after.

As far as mapping names (mueller) to user ids (65611239) it's one of the sillier I've seen.

Kreuzberg

When I establish a socket connection with www.meinewebseite.net using fsockopen() and send a

GET /mueller HTTP/1.0\n\n

request, the response reads

HTTP/1.1 404 Not Found
Date: Fri, 05 Dec 2003 13:42:59 GMT
Server: Apache/1.3.29 (Unix)
Connection: close
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<HTML><HEAD>
<TITLE>404 Not Found</TITLE>
</HEAD><BODY>
<H1>Not Found</H1>
The requested URL /mueller was not found on this server.<P>
<P>Additionally, a 500 Internal Server Error
error was encountered while trying to use an ErrorDocument to handle the request.
</BODY></HTML>

Requesting the above-mentioned "real" URL leads to the same response.

Is my request wrong? How would you open this URL in PHP?

Kreuzberg

OK. One step to the solution is sending a Host header. So instead of

GET /mueller HTTP/1.0\n
\n

send

GET /mueller HTTP/1.0\n
Host: www.meinewebseite.net\n
\n

in order to request http://www.meinewebseite.net/mueller.

This is because the URL is on a virtual server.

fopen() obviously cannot handle this - maybe a bug?

I'm not really familiar with the whole HTTP issue, so sorry if this is a little vague...

superwormy

fopen() sends a default set of headers and really doesn't do the job of opening URLs over HTTP. Use fsockopen() instead and you'll have much better luck.