[RESOLVED] Possible to store the HTML of an external site in a variable?

RonB

Possible to store the HTML of an external site in a variable?

Such as...

$yahoo_finance_html = getpage("http://finance.yahoo.com");

Then you can use that html to extract certain things like stock quotes from the Yahoo! Finance site.

If you can't do it with only PHP, is there any other way to do it? Like using AJAX or something? Thank you.

MarkR

Yes you can do this, but without explicit permission from the site owner:
- Your screen-scraper may get banned at any time with no notice
- The site owner may change the structure so your screen-scraper no longer works
- The site owner may change the content to attempt to exploit your server-side code.

For preference, use a dedicated machine/machine protocol, e.g. a HTTP page designed to be read by machines.

Mark

RonB

A screen-scraper? That really makes it sound bad. Is it really that bad? I made one in Delphi to get user statistics for Age of Mythology and it worked quite well and it just got the HTML source and manipulated it to display the stats.

And you say I can do this, but is there a PHP function able to do this? or do I need to resort to other means like python or something. Thank you.

Ron

bradgrafelman

You can use [man]file_get_contents/man (assuming allow_url_fopen is enabled) to retrieve the HTML, but as MarkR suggests, if you're doing some sort of automated request without permission, who knows what might happen.

RonB

Ahh, file_get_contents helped a lot. Thank you Brad and Mark.

You are right. I will ask permission before attempting it. 🙂