Need to extract data from tables in an html page. How can I do this?

jonasr

I've been trying to extract data from an html page but cannot parse the data very well. The page I have has about four tables but I only need the info from certain table data sections below the main table. It's not XML formatted, so it doesn't look easy. I've found examples of how to get the attributes from tags, but not getting the text from between the tags.

I managed to get the page into a string and redisplay it without tags, but I only need certain pieces. I'd like to be able to put the text between each <td>...text...</td> section into different variables. I haven't found any examples of how to do this kind of thing without already having the html in a distinct XML tagged format. Surely someone has had to do this before. I'd rather not reinvent the wheel if I don't have to.

Can someone please point me to some examples of how to get data from between HTML table data tags? Any help would be greatly appreciated.

Thanks,
Jonas.

planetsim

What exactly do you want done.. Sorry im a bit tired so my concentration on reading is getting low..

Are you trying to get text or display the html file itself..

jonasr

I'm trying to extract the text that is in an html table. ie.
<table>
<tr>
<td>
<font x="blah"><b>Some text </b></font>
</td>
</tr>
</table>

I need to get "Some text" out of the table and into a variable that I can manipulate. I already fopen()ed the page into a string $page. Then split() the page into an array of lines. I'm stepping through the lines and eregi()ng the lines to get what I need. It's really messy and not really working very well. The page I'm looking at has a different number of table elements at different times but always the same number of tables(4). More precisely, I need just the text from between the table data tags.

I was hoping to find an example of parsing the text out from between all the table, tr, td, and misc tags. I just need the text that would be displayed in the html table so I can redisplay it in another style. I don't have access to the database the html page is created from.
There are some extra variables and lines that don't do anything.

Thanks,
Jonas.

Files:
// Begin mls2.php parsing file
<?
error_reporting(8);
require_once("makestring.php");
require_once("inc/output_fns.php");
do_html_header("MLS Test Page");

$in_table = 0;
$in_td = 0;
$table_count = 0;
$table_data = "";
$td_start = array();
$td_end = array();
$edited_page = "";

$page = stripslashes(GetFile("http://rtr/test-mls.html"));
//$page = eregi_replace("<script[^{>]>(.)<\/script>",} "\n Trimmed\n\n ", $page);
$edited_page = eregi_replace(" +", " ", $page);
$edited_page = eregi_replace("Copy", "replaced", $page);
$page = eregi_replace("<script[^{>]>(.)<\/script>",} "\n Trimmed\n\n ", $page);
$page = eregi_replace("<p align=\"center\">\&nbsp[^>]>(.)", "\n Trimmed\n\n ", $page); // replace everthing between two points
echo $edited_page;
//$page = eregi_replace("^Start a ", "\n Trimmed\n\n ", $page);
//$page = stripslashes(GetFile("http://rtr/mls117266.html"));
// echo "<table width=100% border=3>";
//$td_start = eregi_replace("<script[^{>]>(.)<\/script>",} "\n Trimmed\n\n ", $td_start);
$td_start = split ("\n", $page);
//$td_end = split ("<\/", $page);

for ($i=0; $i< count($td_start); $i++){
//print_r(array_values($td_start));
$notags = strip_tags($td_start[$i]);
echo $i;
echo $td_start[$i];
$striped = eregi("Island of", $notags);
echo $striped;

}

 echo $td_start[54]."\n\n";
 echo $td_start[36]."\n\n";
 echo $td_start[37]."\n\n";
 echo $td_start[38]."\n\n";
 echo $td_start[85]."\n\n";
 echo $td_start[95]."\n\n";
 echo $td_start[96]."\n\n";
 echo $td_start[101]."\n\n";
 echo $td_start[112]."\n\n";

?>
</body>
</html>

//

//

<html>
<head>
<title>Hawaii Information Service: real estate, property, MLS and public records, parcel maps, business
registrations, and professional and vocational licenses online for the state of hawaii.</title>

</head>
<body bgcolor="#FFFFFF" background="/images/hiback.gif" text="#333333" link="#339999" vlink="#666666" alink="#999999">
<div align="center">
<p><img src="/images/hihead.gif" width="392" height="15"></p>
<table border="0" width="580" cellspacing="0" cellpadding="0">
<tr>
<td width="293">
<img src="pubimages/smhishead.gif" width="293" height="79" vspace="3">
</td>
<td width="287" align="left" valign="middle">
<ul>

      <li><b><font face="Verdana, Arial, Geneva" size="2">Start
	   a <a href="default.htm">NEW</a> search</font></b></li>
      <li><b><font face="Verdana, Arial, Geneva" size="2">Get
	  <a href="help.htm">HELP</a></font></b></li>

    </ul>
  </td>
</tr>

<tr> 
  <td colspan="2" align="center" valign="top">

<div align="center"><center>
<p> </p>
<table border="1" width="100%" cellspacing="0" cellpadding="3" bgcolor="#FFFFFF" bordercolor="#FFFFFF" bordercolorlight="#999999" bordercolordark="#FFFFFF">
<tr>
<td colspan="4" bgcolor="#339999"><div align="center"><font face="Verdana, Arial, Geneva" color="#FFFFFF"><font size="4"><b>Island of Hawaii,
North Kona District</b></font><br><b>
<font size="3">SEA VILLAGE</b>
</div>
</td>
</tr>
<tr align="center" valign="middle">
<td colspan="2"><font face="Verdana, Arial, Geneva" size="2"><b>MLS #</b>
117266</font></td>
<td colspan="2"><font face="Verdana, Arial, Geneva" size="2"><b>$305,000</b></font></td>
</tr>
<tr>
<td colspan="4">
<div align="center">
<table border="4" cellpadding="0" cellspacing="0">
<tr valign="middle">

  <td align="center"><img border="3" src="http://photos.hawaiiinformation.com/WebPhotos/6/66/117266301.jpg" width="400" height="300"></td>
  </tr></table>
  <p><font face="Verdana, Arial, Geneva" size="1">13 photographs
  are available for this listing.<br><a href="SpecificListing.asp?MLS=117266&MLSEVAL=y&ALLPICS=1">Click here to view available pictures</a>.</font></p>

</div>
</td>
</tr>
<tr>
<td colspan="4">
<div align="center">

	<font face="Verdana, Arial, Geneva" size="2">
	2 bedrooms &amp; 2 baths<br>
	967 sqft of living area

	</font><br>

<font face="Verdana, Arial, Geneva" size="2">
Fee simple
ownership</font>

	</div>
  </td>
</tr>

<tr> 
  <td colspan="4" align="center"><font face="Verdana, Arial, Geneva" size="2"><b>Includes:</b>
  appliance/unit, ceiling fan, carpet, drapes, dryer, disposal, dishwasher, floor tile, furniture/equipment, kitchenware, landscaped, linens, laundry, microwave, in ground pool, refrigerator, range, rock walls, spa, TV cable, washer
  </font></td>
</tr>

<tr> 
  <td colspan="4" align="left"><font face="Verdana, Arial, Geneva" size="2"><b>Remarks:</b>
  Terrific unit completely upgraded. Custom tile in kitchen & baths. Turn key quality furnished. Strong rental. This unit shows like a model! One mile from downtown Kona. This complex features Ocean Front Pool, Spa & Tennis Ct.</font></td>
</tr>

<tr align="center" valign="middle"> 
  <td colspan="4">
  <table border="0" cellpadding="0" cellspacing="0" width="100%">
  <tr><td align="left">

</td><td>
<div align="center">
<font face="Verdana, Arial, Geneva" size="2"><b>Contact: <br>
Robert 
D 
Wilkinson
 RA </b>

	<br><b>E-mail:</b> <a href="mailto:aliib35@yahoo.com?Subject=Inquiry_about_MLS_117266_(H.I.S._Referral)">aliib35@yahoo.com</a>

  </font>
  </td>
  <td>

   <a href="http://www.coldwellaloha.com" target="4018">
<img src="/Images/custom/smlogos/offices/4018.gif" border="0" hspace="10" vspace="10" align="right"></a>


  </td>
  </tr>
  </table>
  </td>
  </tr>

<tr align="center" valign="middle"> 
  <td colspan="4"><font face="Verdana, Arial, Geneva" size="2"><b>Coldwell Banker Aloha Properties - Kona<br>(808) 329-3545</b>

<br><a href="http://www.coldwellaloha.com" target="Coldwell Banker Aloha Properties - Kona">[url]http://www.coldwellaloha.com[/url]</a>

  </font>

  </td>
</tr>

</table>
</center></div>

</td>

</tr>
</table>

<p align="center"> </p>

<p align="center"><font face="Verdana, Arial, Geneva" size="2"><a href="http://www.hawaiiinformation.com/Static/subscribers.asp">REsearch Login</a> |
<a href="Default.htm">Hawaii Property & Information Search</a><br>
<a href="http://www.hawaiiinformation.com/Static/news/newsmain.htm">News</a> |
<a href="http://www.hawaiiinformation.com/Static/products/productlist.htm">Products and Services</a> |
<a href="http://www.hawaiiinformation.com/Static/forms/formsdir.htm">Forms and Documents</a><br>
<a href="http://www.hawaiiinformation.com/Static/about.htm">About Hawaii Information Service</a> |
<a href="http://www.hawaiiinformation.com/Static/contact.htm">Contact Us</a> |
<a href="http://www.hawaiiinformation.com/Static/siteindex.html">Site Index</a> |
<a href="http://www.hawaiiinformation.com/">Home</a></font></p>
<p align="center"><img src="/images/dividerbar.gif" width="300" height="17"><br>
<font color="#006666"><b><font face="Verdana, Arial, Geneva" size="2">Hawaii Information
Service</font></b></font><b><br>
<font face="Verdana, Arial, Geneva" size="2">www.HawaiiInformation.com<br>
</font></b></p>
<p align="center"><font face="Verdana, Arial, Geneva" size="1">MLS & public records, parcel maps, business
registrations,<br>
and professional and vocational licenses online for the State of Hawaii.</font><font face="Verdana, Arial, Geneva" size="2"><br>
<img src="/images/bar.gif" width="580" height="3" vspace="4"><br>
<font size="1" color="#006666">Copyright © 1998-1999 by MLS Hawaii, Inc.<br>
All Rights Reserved.</font></font></p>

<p align="center"><font face="Verdana, Arial, Geneva" size="1" color="#006666">This information
has been supplied by third parties and has not been independently verified<br>
by Hawaii Information Service and is, therefore, not guaranteed.</font></p>

</div>
</body>
</html>

planetsim

I dont think this will help.. but maybe try strip_tags();

that could get rid of all html for yay.. make your job easier.. But then im not sure if it will work.. Have a try who knows.. You may loose 10lines or so

Weedpacket

Originally posted by jonasr
Then split() the page into an array of lines.

First, don't bother doing that - just leave it all as one big string - then you don't have to mess with patterns that sprawl over multiple lines.

I'm stepping through the lines and eregi()ng the lines to get what I need. It's really messy and not really working very well.

There's your big problem: eregi's POSIX regexps are cheap and nasty (at least when compared to PCRE regexps) - use preg_match and preg_match_all instead.

As an example, here's a bit of code that will isolate the full text of the outermost table (assuming that the entire page is in a single string):

preg_match("/<table[^>]*>(.+)<\/table>/is", $page, $matches);

$matches[1] contains the text. With that bit isolated, you can then concentrate on locating appropriate rows, subtables or it is you're after.

Once you've isolated the bits you want it's easy to reassemble them however you please. Yes, it could probably be done with a swarm of calls to preg_replace(), but it sounds like an unreliable and limiting way of goung about it.

jonasr

Thanks, that was helpful. I've been playing with it all day. I can pear down the tables just fine with that, but I still don't see how to extract the text from between the tags.
What happens to the data that gets skipped? When you pear down the tags I cannot find where the data is that got removed. I attempted to use preg_grep to get a string of info from <td colspan="4" ....and the next </td> before doing the next table removing, but I'm not sure where the removed data is stored.

I thought I would be able to pear down to the <td> tags the same way, but it doesn't seem to work the same way. What am I missing?

Tried: $heading = preg_grep("/<td colspan=\"4\"[^{>]*>(.+)<\/td>/is",} $first_table);

I expected $heading to have everything for that bit of table data, but it found nothing.

Also, subsequent preg_match/es eventually break with an error that matches[some number] is invalid.

Current code:
<?
error_reporting(8);
require_once("makestring.php"); // include remote file in one string
require_once("inc/output_fns.php");
do_html_header("Single MLS Lookup Test"); // Create header and title page.

$page = stripslashes(GetFile("http://rtr/test-mls.html")); // remove slashes from string
$edited_page = eregi_replace(" +", " ", $page); // replace multiple spaces with one space.
// replace everything between <script> tags with a space
//$page = eregi_replace("<script[^{>]>(.)<\/script>",} "\n\n ", $page);

  if (preg_match("/<table[^>]*>(.+)<\/table>/is", $page, $match)) {
  $first_shell = $match[1];
  }

  echo "<h2>Begin 1-B extract.</h2>\n";
  if (preg_match("/<table[^>]*>(.+)<\/table>/is", $first_table, $match)) {
  $first_table = $match[1];
  echo $first_table;
  echo "<h2>End 1-B extract.</h2>\n";
  } // outside table tags removed


  echo "<h2>Begin 2 extract.</h2>\n";
  if (preg_match("/<table[^>]*>(.+)<\/table>/is", $first_shell, $match)) {
  $second_shell = $match[1];

  echo $second_shell;
  echo "<h2>End 2 extract.</h2>\n";
  }

//print_r(array_values($match));

</body>
</html>

Weedpacket

Any bits you match that you want to keep, wrap () around them and their values will be in the $matches array (assuming that's you're calling the third argument to preg_match()).

Note also that preg_match() will only match the first instance of the pattern it finds - if you want to find all matches in the string, use preg_match_all() instead.