Looping through a site and outputting to new files

chymoe

If anyone could shed some insight or point me in the right direction...

I'm looking to use the mini script below which is essentially an RSS script. I now want to tell this to spider/loop through my entire site (like winhttrack or pagesucker) and then output the $content of each to it's own file.

<?php

// Get the page you want to parse
$url = "http://www.example.com/test.html";
$data = implode("", file($url)); 

// Get all content between <body> and </body>
preg_match_all ("/<body>([^`]*?)<\/body>/", $data, $matches);

// Loop through the page
foreach ($matches[0] as $match) {

// Get Content between your comment lines
preg_match ("/<!-- start comment-->([^`]*?)<!-- end comment-->/", $match, $temp);
$content = $temp['1'];
$content = strip_tags($content);
$content = trim($content);

// Print Content Found
echo $content;


}

?>

Thanks!

dalecosp

Well, assuming that there are any anchor tags in the data you're reading, I suppose you'd use more preg or ereg stuff to isolate the names of the pages, store them in an array, and then have the spider routine do something similar with each of them.

Of course, I've never written a spider before. But, I'd guess this is at least a "point" in "the right direction".