search for urls with file_get_contents()

mechew

I'm trying to create a real simple proxy server and this is what I have sofar:

$url = $_GET['url'];
$page = file_get_contents($url);

$page = str_replace("<a href='", "<a href='http://www.myurl.com/?url=", $page);
$page = str_replace("<a href='/", "<a href='".$url."/", $page);
$page = str_replace('<a href="', '<a href="http://www.myurl.com/?url=', $page);
$page = str_replace('<a href="/', '<a href="'.$url.'/', $page);

header("Cache-Control: no-cache, must-revalidate"); // HTTP/1.1
header("Expires: Mon, 26 Jul 1997 05:00:00 GMT"); // Date in the past

echo $page;

How can I go through $page and replace all the links with http://www.myurl.com/?url=$thelink ?

So when the page comes up I can just click the link and it will just send that to my proxy server?

What I have works great as long as the links either start with [url]http://,[/url] www, or /. But if its a local page this won't work.

MarkR

This is a nontrivial task.

Your best bet would be to use a HTML parser. Regular expressions are not sufficiently powerful to parse HTML in the general case, and you'll end up messing the page up.

Fortunately PHP has a HTML parser linked into it by default, in DOM::loadHTML. Be mindful of encodings when using this.

You can then use the DOM methods to identify anchor links (plus lots of other things as necessary) and adjust their hrefs.

There are a lot of other pieces of coding which are required which are nontrivial - one of which is relative URL resolution. This is extremely complicated- consider the following relative URLs:

test.html
./test.html
../test.html
/test.html
../../test.html
././../test.html
#blah
?x=42

All of which are valid and have a specific meaning. BASE HREF elements also need to be considered, along with many other things.

Mark

larshemel

The problem is most often that you are either using file_get_contents in a for or while lus and in that way accessing it too often, or you have st your DNS settings incorrectly. Try to change localhost with 127.0.0.1 and if it works you know that you have to update your DNS.

More info can be read on http://www.larshemel.com/forum/timeout_file_get_contents_localhost_url .

Cheers.