My background: Engineer with coding experience; mainly data handling with algorithms. I have a decent grasp of PHP now and I am trying to learn cURL.

I have been successful posting to forms like mapquest.com and other simple forms using the following code.

<?php
    $curl = curl_init();
    curl_setopt($curl, CURLOPT_URL,"http://www.mapquest.com");
    curl_setopt($curl, CURLOPT_POST, 1);
    curl_setopt($curl, CURLOPT_POSTFIELDS, "address=Cherry Street");
    curl_exec ($curl);
    curl_close ($curl);
Print $curl:
?>

This code does not work for all sites though. Like HTTPS sites and username/password text fields from sites like ebay, yahoomail, and online banking.

I do not fully understand the physics of how data is passed along form server to server and thus can not understand why this code works sometimes and sometimes doesn’t.

For Https sites I found some curl code that initializes curl, pulls the cookies from the header and stores them in cookiejar, closes curl, initializes curl again, uses the cookies on the next page and post the values to the form. (By the way I can’t get this to work)

My questions:

-Why do I need to pull the cookies from the index page and pass them to the login page?

-They also use the referrer option. Why?

-Why can I use the above code to post field on mapquest.com and can’t use the same code to post fields on mail.yahoo.com, ebay, bankofamerica.

If someone could point me to a guide for dum-dums to cURL and HTTP I would appreciate it and maybe just understanding the physics of how this stuff works.

Thank you very much!
rfulky

    You'll need SSL support enabled on your web server to connect via HTTPS (which is used by eBay, Bank of America, and so forth). (I don't know whether SSL support is enabled by default in Apache; it probably depends on which version you have installed.)

    You may also need to play with the various SSL options in curl_setopt(). And I don't think you would need to purchase and install a secure certificate, but I could be wrong about that.

    As for the physics of passing data, it's pretty simple, really: it's all just embedded in the HTTP header. Every HTTP connection has a header, whether it's between a web browser and a server, or between two servers. That header may, or may not, include POST data.

    I don't fully understand your questions about cookies. Do you have some sample code you could post?

    Most sites authenticate users using cookies, which are placed on the client at login. Whenever you try to view a restricted page (such as the forms you're trying to post to), the remote server checks the client to make sure a valid cookie is present. So you'll have to make sure the necessary cookie has been embedded in the HTTP header before you initiate the cURL connection.

      Write a Reply...