Php Web Crawler

[mclarko]

Does anybody know where I can get hold of some code to traverse a website looking for urls (could be anything other than a url of course).

Cheers!

Malc

[mclarko]

Originally posted by mclarko
Does anybody know where I can get hold of some code to traverse a website looking for urls (could be anything other than a url of course).

Cheers!

Malc

Forgot to mention - I'd like to parse the pages by perhaps using file() and then use ereg to find an instance of o URL, but just wondered if some jazzed up code existed already. Programmers are lazy right!!? ;-)

[rontgoff]

I have one that I use for http://conveyorgroup.com/sprock But it was difficult to come up with and is a on going work in progress. I can't just let go. But It is very do able. If you get stuck on something I could help. But try first. You might want to look for things on this forum or even on google to find how many urls are on a page(thats how I got started).