pulling data from HTML tables

jadeddog

hey there... im looking for a way to pull information from HTML tables.... basically i want to be able to data mine certain statistical websites (hockey stats to be more specific) by stripping out the information contained in their HTML tables, and then entering this information into a MySQL database

ive looked and looked and looked, and i cannot find a program that does this.... does anybody have any ideas?

thanks in advance

bradgrafelman

Data mining is hard because:

You have to make sure it's not forbidden by the remote site, and
If the remote site alters their format, even slightly, it can easily break your code.

Having said that, you can always retrieve the remote website ([man]file_get_contents/man does a good job) and use some [man]PCRE[/man] functions to grab the data.

A much better solution would be to look for an XML/RSS type of feed. If you can't find one... contact the site and ask if they have one or would set one up for you.

jadeddog

bradgrafelman wrote:
Data mining is hard because:

You have to make sure it's not forbidden by the remote site, and

If the remote site alters their format, even slightly, it can easily break your code.

Having said that, you can always retrieve the remote website ([man]file_get_contents/man does a good job) and use some [man]PCRE[/man] functions to grab the data.

A much better solution would be to look for an XML/RSS type of feed. If you can't find one... contact the site and ask if they have one or would set one up for you.

these PCRE functions can grab the rows/columns of tables?

bradgrafelman

Using a function such as [man]preg_match/man with a crafted regexp, you can do just about any kind of data mining you want. But, as I said, even a slight change in format would break your regexp patten.

Weedpacket

You could also use the DOM extension if you have it to load and parse the HTML; it's somewhat more robust than using regular expressions, but you'll still be at the mercy of the slightest design change. Really, as bradgrafelman says, find a source that is suitable for machine consumption and use that; if the site's owners wanted machines to be collecting stats from the site, they'd provide such a thing.

jadeddog

thanks for the tips.... yeah, i realize that the slightest change in format will screw things up, but thats ok really.... the site im pulling from hasnt changed its format in years, and i dont really care if they do anyhow.... im just doing this for fun really, and because i find it interesting , so if it stops working one day, it wont be a big deal at all

im not familiar with the DOM extension, but what i have found on the net seems to state that its a parser for XML documents.... will it work for regular html documents as well?

Weedpacket

If you look at PHP's documentation for its DOM extension, you'll note a loadHTML() method.

nuttynibbles

hey i was looking on extracting info from a table and i found this article.. what im doing is similar but i cant seems to extract the hyperlink out using DOM.. any idea guys?