Webpage Parser

rayearth

I'm not exaclty sure if this is the correct title as I'm not completely sure how I should go about tackling this problem.

Anyway, I currently have a bunch of static HTML pages. What I want to do is make the server automatically search for some strings and replace them with a predefined link that would be read from a mySQL database. If possible, check to see if the text is already part of another link, if it is, then ignore that entry and go on to the next one.

The tricky thing with this problem is I need the modification to be done only when the pages get requested as the links from the database is always subject to change and I do not want to keep 2 copies of the whole page (the original and processed versions) as that increases the chances for things to mess up.

Anyway help/suggestion would be greatly appreciated. Thanks in advance.

feldon23

Rewrite the HTML pages as PHP, then do the queries inside the pages.

What you describe is exactly what PHP is for-- on-demand content inclusion/generation.

rayearth

But doesn't that mean I have to go update every single page each time a new keyword for a link is added? Or is there a way in PHP to make it do that automatically?

feldon23

But doesn't that mean I have to go update every single page each time a new keyword for a link is added? Or is there a way in PHP to make it do that automatically?

I'm a little confused here.

If you change all the pages to PHP (a simple matter of renaming the files to .php, it is not even necessary that the file has PHP code in it for it to display properly), then you can progressively insert more and more dynamics into each page.

If every page needs to link to a certain part of your website whose address might change, then on each of the pages in your website, you'd add this:

<?php
//code to grab a URL from MySQL and put it in $url_to_area1
//code to grab a URL from MySQL and put it in $url_to_area2
?>
<html>
.
.
. (normal website here)
.
.
.
<a href="<?php echo $url_to_area1; ?>">Click here</a> to go to Area 1.
<a href="<?php echo $url_to_area2; ?>">Click here</a> to go to Area 2.
</html>
?>

If you have a certain group of an unknown number of links that will be displayed as a cluster or maybe an html table, then you could do a slightly more advanced PHP script like so:

<?php
//code to grab the global list of all the dynamic links from MySQL
//read it into an array $glob_links, then read the global list of
//descriptions FOR the dynamic links from MySQL and read it into
//the array $glob_links_desc

reset($glob_links_desc);
i=0;

echo "<table><tr>";

foreach ($glob_link as $gl) {
i += 1;
echo "<td><a href=\"".$gl."\"><b>".$glob_links_desc[i]."</b></a></td>";
}

echo "</tr></table>";
?>

You could also have a whole chunk of each page that is dynamic.

To do this, you'd have your ordinary HTML pages renamed to *.php and then insert code in there.

<html>
<head>
.
.
.
.
<?php include "dynamiclinks.php"; ?>
.
.
.
</html>

Then if you change the contents of dynamiclinks.php, everywhere it is displayed (all the other pages), it will be up-to-date.

rayearth

Thanks for the input feldon23.

Ok, let me try to explain it in another way. Let's say I have a whole blob of text. What I want to do is have something that searches through this, looking for keywords and replacing them with a link to another page.

If I have the following text:

I have cats and dogs

If my list of keywords only includes "cats", then the script would replace the text with:

I have <a href="catlink">cats</a> and dogs

If I add the keyword "dogs" to the script, then it would produce:

I have <a href="catlink">cats</a> and <a href="doglink">dogs</a>

Now if I remove the "cats" keyword, it would produce this instead:

I have cats and <a href="doglink">dogs</a>

I know I could just use a simple find and replace on any text editor to do this, but my list of keywords that needs replacing is always increasing and the links are always changing. I know using PHP would solve the link changing problem (such get the link from a database) but it's the searching for keywords through each page that I'm stuck on.

I not only have one page that needs this processing, but a whole site of pages. That's why I think it might be better to have a central script that does all the work instead of having a copy of it on each page as it would make updating a lot easier that way.

I hope this made sense...

feldon23

All of your HTML documents:

<?php
$data=<<<QQQ
<html>
.
.
.
.
.
</html>
QQQ;

include "keywordlinks.php";
?>

keywordlinks.php

<?php

//mysql code to read array of keywords into $keys
//mysql code to read array of URLs into $key_urls

//or you could just write the keywords and URLs (links) right
//into the PHP if you wanted like:

$keys  = array(1 => 'dogs', 'cats', 'elvis', 'horseshoes');
$key_urls = array(1 => 'http://www.dogs.com/', 'http://www.garfield.com/', 
      'http://www.graceland.com/', 'http://www.closeenough.com/'_;


reset(i);

foreach ($keys as $key) {
  i += 1;
  $searchstr = "<a href=\"".$key_urls[i]."\">".$key."</a>";
  $data = str_replace ($key, $searchstr, $data);
}

echo $data;

?>

rayearth

Thanks!