Get/extract info from webpages!!?

Guldstrand

Hi all!

First of all, i just want to say sorry if i´ve posted this in the wrong forum.

Now to my question...
I´m pretty fresh on php and mySQL, but i really need some help with a code or script.

Is there a way to get/extract specific info from a webpage, and then save it in specified sql-tables in my database!!?

Yes i know.. i really suck in explaination. 🙁

Let´s state an exampel...
I want to get all the info (plot, titel..) from ie imdb.com, and then save it into my own database.

edwardsbc

You can get the contents of a web page into your scripts by using the "CURL" functions. Once you have the contents as text, however, it will be an exercise in regular expressions to extract the components you want.

If your source can produce an XML reply, that would be a simpler way to go so far as parsing. PHP has XML libraries to assist you with XML parsing.

TenFold

just use

$someVariable = file_get_contents("http://www.imdb.com");

and then all the source of http://www.imdb.com will be in $someVariable

MarkR

Firstly, you should get permission from the owners of the data that you're allowed to syndicate it.

Most sites which allow you to syndicate their data provide a feed in a more convenient (and static, non-changing) format than a HTML page.

In the unlikely event that you can legitimately screen-scrape IMDB, I'd say you'd probably want to use something like DOM->loadHTML to parse it, then step through the nodes to find the bit you want (either by ID, or by ancestry).

Pay special attention to the encoding of the page and ensure that DOM interprets it correctly. DOM always handles all its output in UTF8. The input could be in any supported encoding as long as you either tell it what it is, or it can work it out (for example, from a html meta tag)

http://uk.php.net/manual/en/function.dom-domdocument-loadhtml.php

Mark