Several of my pages on one of my sites requires the user to accept the privacy policy / terms of use. When they click "I accept", a cookie is set, then they are allowed to view the pages.

My problem is, it does not seem as though robots from search engines are allowed to view the pages, because they can not "accept" a cookie.

Is it possible to allow robots to index and continue on to the pages without having to accept the terms of use?

I've been told by others to look at the $_SERVER variables, but I'm not sure how to do this, not to mention how to integrate it.

<?php 
if ( !isset($_COOKIE['masondisclaimer']) || $_COOKIE['masondisclaimer'] != "yes" ) 
{ 
   header ( "Location: mysite.com" ); 
   exit; 
} 
?>

    After asking this question on another board, I received this code, which seems to work...guess I'll have to wait a little while to see if the crawlers actually index the pages now.

    // use lower case only
$robots = array(
   'google',
   'msn',
   'inktomi',
   'excite',
   'alexa',
   'bot',
);


$user_agent = strtolower($_SERVER['HTTP_USER_AGENT']);

foreach ($robots as $robot) {
   if (false !== strpos($user_agent, $robot)) {
       $is_robot = true;
       break;
   }
}


if (!isSet($is_robot)) {
   // do your cookie/redirect code here
}
?>
      Write a Reply...