[RESOLVED] Can't scrape an ASP POST page

searchMOTORHOME

I'm trying to scrape the content from:

http://www.westcountry-motorhomes.co.uk/new_list.asp

Through a browser this page is shown when you've POSTed a form from:

http://www.westcountry-motorhomes.co.uk/new_search.asp

I have permission from the website owner to do this scraping and post the results on my website http://www.searchmotorhome.com

I'm trying to use the following code, but can't understand why it doesn't work. All I get is the html from the new_search.asp code. My hosting company has opened the firewall to the site.

<?
$myURL="www.westcountry-motorhomes.co.uk/new_search.asp";
$chttp = curl_init();
curl_setopt($chttp, CURLOPT_URL,"http://$myURL");
curl_setopt($chttp, CURLOPT_POST, 1);
curl_setopt($chttp, CURLOPT_POSTFIELDS, "vehicleType=All&make=Any&berths=No%20Minimum&priceRange=No%20Limit");
curl_exec ($chttp);
curl_close ($chttp);

var_dump($chttp);
?>

I've seen some forums talk about viewstate - but don't understand how I would implement that or if I need to. Fairly inexperienced in PHP.

Thanks in advance for your help.

cahva

You are sending the form to wrong url. Remember that the form is handled in new_list.asp, not new_search.asp.

So change the $myURL to "www.westcountry-motorhomes.co.uk/new_list.asp"

Also I noticed that you have wrong values in your postfields. Correct one is:

curl_setopt($chttp, CURLOPT_POSTFIELDS, "vehicleType=all&make=any&berths=1&priceRange=999999");

searchMOTORHOME

cahva you are a genius - thank you so much - works like a treat with your suggestions!

Thanks again.

bradgrafelman

Don't forget to mark this thread resolved.