[RESOLVED] Detecting a Spider / Crawler / Search Engine

[partypete]

Hello,

I have made a visitor counter for a site that adds a record to the database each time a page is viewed. To make it a little more accurate, I was wondering if there is a way to detect if the page is being viewed by a search engine spider instead of a human? That way I could use a condition to not execute the database update if the visitor is a spider... or mark the record "Spider".

A stats program can tell you which spiders visited and how often, so I assume there must be some way that I can detect them in my pages.... ?

Thanks for any and all help!

Peter

[drew010]

i cant imagine that you are getting hit by search engines so much as to make a big impact on your counter, but you can check the $_SERVER['HTTP_USER_AGENT'] variable to check if its one of the search engines. each one identifies as a certain name, for example google identifies as Googlebot. You can get a pretty good list of the names here: http://www.jafsoft.com/searchengines/webbots.html#search_engine_robots_and_others

[partypete]

Thanks drew010.

Yeah you're probably right about the impact on the counter. I probably won't add spider detection but it's good to know there is a way if I change my mind.

Peter