curl connection problem

alfredopacino

I'm new here so hi to all community!

I need a little help with a 'simple' curl post connection.
here my test code, nothing special just init and exec a curl.
But the printed result variable contains the search form, not the result page, so i think there's some kind of cookie protection with a server side check/redirect.
please help me with this 🙂

$url = 'http://www.infoleges.it/Dogi2005/default.aspx';

$ch = curl_init();
$fields = array(
    'orderby' => urlencode(1),
    'MaxDocPagina' => urlencode(20),
    'interodocumento' => urlencode(),
    'PrimoElemento' => urlencode(),
    'SecondoElemento' => urlencode(),
    'Titolo' => urlencode(),
    'Rivista' => urlencode(),
    'idRivista' => urlencode(),
    'AnnoPubblicazioneFrom' => urlencode(),
    'AnnoPubblicazioneTo' => urlencode(),
    'Fascicolo' => urlencode(),
    'Tipologia' => urlencode(),
    'tabtesti_testo' => urlencode(),
    'Ambito' => urlencode(),
    'Sezione' => urlencode(),
    'NomeFonte' => urlencode(),
    'Anno' => urlencode(2004),
    'Mese' => urlencode(),
    'Giorno' => urlencode(),
    'Regione' => urlencode(),
    'Citta' => urlencode(),
    'Stato' => urlencode(),
    'Numero' => urlencode(40),
    'Articolo' => urlencode(),
    'Fonte' => urlencode(),
    'AreaGiuridica' => urlencode(),
    'Classificazione' => urlencode(),
    'doSearch' => urlencode("Cerca nell'archivio DoGi")
);
foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; }
rtrim($fields_string, '&');

//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);

curl_setopt($ch,CURLOPT_POST, count($fields));
curl_setopt($ch,CURLOPT_POSTFIELDS, $fields_string);
curl_setopt($ch,CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.6 (KHTML, like Gecko) Chrome/16.0.897.0 Safari/535.6'); 
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file);
curl_setopt($ch, CURLOPT_AUTOREFERER, true); 
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); 		




//execute post
$result = curl_exec($ch);
echo '<pre>';
$doc = new DOMDocument();
@$doc->loadHTML($result);
print_r($doc);
echo '</pre>';

dalecosp

Welcome to PHPBuilder!

(*Don't shoot me if I'm wrong!!! #Scarface) 😃

Hmm. I did some work on something similar once.

IIRC, you had to read the response, get some "magic strings" from the server response, and include them when POSTing to whatever the next page (in this case, search results?) was ...

alfredopacino

dalecosp;11041867 wrote:
IIRC, you had to read the response, get some "magic strings" from the server response, and include them when POSTing to whatever the next page (in this case, search results?) was ...

i know thats why i'm here 🙂
i can't find any magic strings.
there is a encrypted _VIEWSTATE asp.net..you can see that doing a search in that url..

alfredopacino

if you try to do a search (for testing just filling the fields "numero"=40 and "anno"=2004) you can see a

Cookie:ASP.NET_SessionId=fvdcqvzg1uiinc2wrjyaw245

i guess it's part of the problem.

dalecosp

Busy as a one-armed paperhanger here on the day before a holiday, but VIEWSTATE is one of them, I'm pretty sure. The others were EVENTTARGET, EVENTARGUMENT, and EVENTVALIDATION, but _EVENTARGUMENT was null in my case (I think). YMMV, of course. Sorry that I know little about ASP programming...

alfredopacino

thanks for the answer, i evet tried to catch the cookie and send it with the entire POST string in a second curl_exec.
nothing change :mad:

$url = 'http://www.infoleges.it/Dogi2005/default.aspx';

$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);

$returnData = curl_exec($ch);
$cookie = '';
$pattern = '/Set-Cookie:(.*?)\n/';
if (preg_match($pattern, $returnData, $result))
$cookie = $result[1];

curl_close($ch);


$ch = curl_init();
$fields_string='__EVENTTARGET=&__EVENTARGUMENT=&__VIEWSTATE=dDwtMTI1ODA0MzgxO3Q
//CUTTED ABOUT 10k chars
&__start=&__maxpage=&orderby=1&MaxDocPagina=20&interodocumento=&PrimoElemento=&SecondoElemento=&Titolo=&idRivista=&AnnoPubblicazioneFrom=&AnnoPubblicazioneTo=&Tipologia=&tabtesti_testo=&Ambito=&Sezione=&NomeFonte=&Anno=2004&Mese=&Giorno=&Regione=&Citta=&Stato=&Numero=40&Articolo=&Fonte=&AreaGiuridica=&Classificazione=&doSearch=Cerca+nell%27archivio+DoGi';
//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_URL, $url);
curl_setopt($ch,CURLOPT_POST, 31);
curl_setopt($ch,CURLOPT_POSTFIELDS, $fields_string);
curl_setopt($ch,CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.6 (KHTML, like Gecko) Chrome/16.0.897.0 Safari/535.6'); 
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_AUTOREFERER, true); 
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); 		




//execute post
$response = curl_exec($ch);

$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
$header = substr($response, 0, $header_size);
$body = substr($response, $header_size);
echo $body;

dalecosp

Have you tried setting up an array instead of a string ($fields_string)?

The variable name is the key and the values as shown in the POST request.

alfredopacino

well the CURLOPT_POSTFIELDS wants a string as arg, so i can build the array and then implode it (like the first post code) or i can send a megastring like the second code, but i think that's not the problem..:bemused: