Unable to get HTML code from webpages

dougmcc1

I'm having trouble getting PHP to connect to my webpages so I can do link checking. I need to get the title because some pages might work even though they're an internal search result that doesn't return any results (I want to count these as broken links) in which case the title would contain "No items found...".

Here is the code I'm using:
$html = strtolower(implode('', file_get_contents("http://www.yahoo.com")));
echo $html;

I also tried:
$html = strtolower(implode('', file("http://www.yahoo.com")));
echo $html;

And:
$html = strtolower(implode('', fopen("http://www.yahoo.com",'r')));
echo $html;

None of these work. In php.ini allow_url_fopen is set to "On" (I also tried setting it to "1" but that didnt work either). I'm at a loss on this one, any ideas would be greatly appreciated.

Installer

Turn on error checking/reporting and you'll save yourself a lot of time.

You're using "implode()" wrong.

dougmcc1

The only reason implode isn't working is because there is nothing in $html.

And the reason there is nothing in $html is because the script isn't connecting to the webpage. I also tried other scripts I found on the internet that do the same thing with different code and they didnt work either.

This script is running on an internal website which is on a network and behind a firewall, I assume this is the real cause of the problem although I see no settings that need to be changed.

Anyone have any ideas on what I need to change?

bpat1434

First off... as per the PHP manual...

Tip: You can use a URL as a filename with this function if the fopen wrappers have been enabled. See fopen() for more details on how to specify the filename and Appendix M for a list of supported URL protocols.

Now, on to my other questions/observations:
Why are you imploding the HTML source to a webpage? You're not getting an array of items back (so technically you're not using implode right). So what is the point of you imploding a non-array with no separater?

Why are you going to make the HTML tags lowercase? It really shouldn't matter, unless you plan on doing some RegEx or manipulation; however, even then you can do case-insensitive searches/replaces.

What is it you're trying to achieve? A pseudo-proxy? Or are you just doing this for fun?

~Brett

Installer

Your code works fine for me, without the implode. With it, the error message

Warning: implode() [function.implode]: Bad arguments. in C:\htdocs\test.php on line 3

is displayed. I stand by my first reply.

dougmcc1

The original code I posted which uses the file() and implode() functions works in a script I'm running on a different server for a different website. Cant remember why I thought it was necessary at that time - I'd try it without implode but I dont have access to that server right now.

Back to this script for the internal website, when I take out the implode function this is the error message I get for all three code examples:

Warning: file_get_contents([url]http://www.msn.com):[/url] failed to open stream: Bad file descriptor in C:\sokkit\site\myfile.php on line 348

Warning: file([url]http://www.yahoo.com):[/url] failed to open stream: Bad file descriptor in C:\sokkit\site\myfile.php on line 350

Warning: fopen() expects at least 2 parameters, 1 given in C:\sokkit\site\myfile.php on line 352

Hope this helps. Thanks for the replies so far.

bpat1434

Read the manual man!! [man]fopen/man:

fopen ( string filename, string mode [, bool use_include_path [, resource zcontext]] )

<?php
$url= 'http://www.yahoo.com';
$conn = fopen($url, "r");
$html = file_get_contents($url);
fclose($conn);
?>

~Brett

dougmcc1

Please be polite. I tried that code on my own and didnt get any output, your code also doesn't produce any output when I echo $html.

Thanks anyway, anyone else got any ideas? I dont think it's a coding issue, but a network/firewall/configuration issue.

Installer

All of these should work:

$html = strtolower(file_get_contents("http://www.yahoo.com"));
echo $html;

$html = strtolower(implode('', file("http://www.yahoo.com")));
echo $html;

$fp = fopen("http://www.yahoo.com", 'r');
$html = '';
while (!feof($fp)) {
  $html .= fread($fp, 4096);
}
fclose($fp);
echo $html;

If they don't work for you, then I'd agree it's a n/f/c problem

dougmcc1

Yeah none of those worked. Thanks anyway, I'll try another forum.

bpat1434

I was being polite. But when you say "I get the following error:
Warning: fopen() expects at least 2 parameters, 1 given in C:\sokkit\site\myfile.php on line 352"
How are we to know you did or did not read the manual?

Have you tried specifying the port to use? http://www.yahoo.com:80?

It seems odd that you wouldn't be able to "surf" the net from your server (wouldn't that mean you couldn't connect to websites on the server?).

I apologize for my rudeness; it was not meant to be portrayed that way. it was meant more as a comical remark.

~Brett

jayant

Are you normally able to access Yahoo.com using a web-browser from that machine without using a proxy?
If no, you now know its not the problem with the code... 🙂
If yes, then there is some problem in the code... (PS: the code seems fine to me)

jayant

btw while you are doing that, also try using [man]fsockopen/man and its function to connect.
they should work in all cases given the appropriate parameters (even with a proxy)...

LordShryku

doug, are you behind a http proxy server?