Hi

Is there a way to detect if it's a search engine crawler visiting a site ?

What I was looking for is a way to have a text line only to be shown when it's a "normal" hit. And if it's a crawler this line should not be shown.

    Here is a technical document that lays out a few possible web robot tests that you might use to determine if the hit you are getting is a web robot.

    Good luck!

      Sound a bit complicated ?
      When a web statistic shows how many hits a site has from google bots, yahoo bots and so on, does it really detect mouse movement etc. to get this info ?

        What about cookies ?
        Does web crawlers set cookies ?

        I have a cookie set on the site I use to count new visiters.
        When a visiter visits the site for the first time it sets a cookie and add 1 to a database. The next time they visit the site check if there is a cookie, and if it's already there the visiter is not count again.

        Can I use this cookie maybe.
        After SET Cookie check if it's there, and if it's not there it could be a crawler ?
        I don't think many "normal" users have set cookies to disable.

          Write a Reply...