I wrote a script that uses the snoopy class to retrieve info from a remote site that is on a secure server. I rewrote part fo the snoopy calss to use curl to get through a proxy and to do the login into the secure server.

When I am retrieving info from the remote site I first connect to a page where I get particular info from that page and use that information to build a string in a url to call another page on the remote site. When my script calls the other page I get the following error message.

400 Bad Request Bad Request Your browser sent a request that this server could not understand. The request line contained invalid characters following the protocol string.

I commented out the first part of the script and created some variables to replace the ones that are retrieved from the 1st url and I did not get the error message. I also have been able to connect to the 2nd url using a test script.

Does anyone know why it won't connect to the second url?

Could it be that I need to hide the fact that I am logging in multiple times. The remote site uses htaccess to do the login and I had to use curl to be able to login.

This is the function I use to call the snoopy class.
// function fetch_text()

function fetch_text($URL, $snoopy){
     if($snoopy->fetchtext($URL)){
          $result = $snoopy->results;
		  return $result;
	 }
     else{
	      // create the to address
		  $to = "email@example.com";
	      // create the subject of the email
		  $subject = "Error fetching info from manheim";
	      // And, last (before we build the email), set up some mail headers
          $headers  = "From:  <" . $storeEmail . ">\r\n";
		  $headers .= "Reply-To: <" . $storeEmail . ">\n";
          $headers .= "X-Sender: <" . $row["email"] . ">\r\n";
		  $headers .= "Message-ID: " . date("r") . $row["email"] . "\n";  
$headers .= "Return-Path: <" . $storeEmail . ">\r\n"; $headers .= "X-Mailer: PHP Mailer 1.0\n"; $headers .= "Date: " . date("r") . "\n"; $headers .= "Content-Type: text/html;charset=iso-8859-1\n"; $headers .= "MIME-Version: 1.0\n"; // create error message $message = "error fetching document: " . $snoopy->error . "\n"; // send the email mail($to, $subject, $message, $headers); } }

This is the part of the snoopy class I rewrote so that I can use the curl function instead of the curl binary.

This decision does the login.

if(!empty($this->user) || !empty($this->pass)){
		    $ch = curl_init();	
			$headers[] = "Authorization: BASIC " . curl_setopt($ch, CURLOPT_USERPWD, $this->user.":".$this->pass);//.base64_encode($this->user.":".$this->pass);
		}

This is the curl to get through the proxy, etc.

curl_setopt($ch, CURLOPT_VERBOSE, 1);
        curl_setopt ($ch, CURLOPT_PROXYTYPE, CURLPROXY_HTTP);
        curl_setopt ($ch, CURLOPT_PROXY, "http://proxy.shr.secureserver.net:3128");
        curl_setopt($ch, CURLOPT_POST,1); 
        curl_setopt($ch, CURLOPT_POSTFIELDS,$body); 
        curl_setopt($ch, CURLOPT_URL,$safer_URI); 
        curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2); 
        curl_setopt($ch, CURLOPT_USERAGENT, $this->agent); 
        curl_setopt($ch, CURLOPT_RETURNTRANSFER,1); 
        curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); // this line makes it work under https 
        curl_setopt($ch, CURLOPT_HEADER, 1); 
        curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); 
        curl_setopt($ch, CURLOPT_TIMEOUT, 30); 

    curl_setopt($ch, CURLOPT_REFERER, $this->referer); 
    curl_setopt($ch, CURLOPT_COOKIE, substr($cookie_str,8)); 

    $results= curl_exec ($ch); 
    $return = curl_error($ch); 

    list($response_headers,$response) = explode("\r\n\r\n",$results,2);  
    $response_header_lines = explode("\r\n",$response_headers); 
    curl_close ($ch);

	if($return)
	{
		$this->error = "Error: cURL could not retrieve the document, error $return.";
		return false;
	}

    Here is the part of the script that I am having problems with.

    while ($row = @ mysql_fetch_array($result)){
    	           if (!isset($startNum))
    		            $startNum = 1;
    			   echo "startNum" . $startNum . "<br>";
    		       while ($startNum <= $numFound){
    		            // create url
    					echo "auction date" . $auctionDate . "<br>";
    					$auctionID = $row['auction_id'];
                        $URL2 = "https://www.domain.com/members/presale/control/powersearchDetail?PS=&orderby=&region=&saleDate=" . $auctionDate . "&cid=&locationState=&notebookaction=&showopts=&notebook_sale_channel=&origsalechannel=1&auctionID=" . $auctionID . "&totalNumResults=1&notebook_item_id=&cyberlot=&rowIndex=&showoptions=&saleNumber=&group=&salechannel=1&appUrlId=1007&fromyear=" . $fromYear . "&numresultsperpage=1&app_id=1&pageNumber=" . $numFound . "&resultsPerPage=1&navigation=&saleID=&notebook_sale_id=&year=" . $fromYear . "-" . $toYear . "&toyear=" . $toYear . "&notAvailableCount=0";
    			        // fetch text from remote site
    			        $text = fetch_text($URL2, $snoopy);
    					echo $text . "<br>";
    			        // get the year, make and model of the car  	 
                        $carInfo = get_between($text, $asterisk, $vin);
    			        $carInfo = explode(" ", $carInfo, 3);
    					echo "year: " . $carInfo[0] . "<br>";
    					echo "make: " . $carInfo[1] . "<br>";
    					echo "model: " . $carInfo[2] . "<br>";
    			        // get the rest of the vehicle info
    			        $vinNum = get_between($text, $vin, $color);
    					echo "vin: " . $vinNum . "<br>";
    			        $carColor = get_between($text, $color, $mileage);
    					echo "color: " . $carColor . "<br>";
    			        $carMileage = get_between($text, $mileage, $location);
    					echo "mileage: " . $carMileage . "<br>";
    			        $needle = totalStr($text, $conditionReport, $i = 0);
    
    		        if ($needle == 0)
    		             $s2 = $marketReport;
    		        else
    		             $s2 = $conditionReport;
    
    		        $carOptions = get_between($text, $options, $s2);
    				echo "options: " . $carOptions . "<br>";
    		        $location = $row["auction_id"];
    				echo "location: " . $location . "<br>";
    				echo "sale date: " . $queryDate . "<br><br>";
    
    				// increment the counter 
    			    $startNum++;
    	       }
          }
      Write a Reply...