Protocol Wrappers

[ferritt]

I have been trying to write a block of code that will retrieve collect data from a website. One of my biggest chalenges is that using cURL for imitating a browser client is not possible as it not installed universally on the webservers where the code will run.

I have been successful in reproducing the funcionality, including dealing with POST data, and handling cookies, so i got that going for me... which is nice.

In short, the code uses fopen() and stream_get_contents() to retrieve the pages. The problem i'm running into is that the contents of the resulting stream is not consistant across different platforms.

For example, on my laptop where i am doing much of development, the contents of the stream meta data array is similar to the following:

Array
(
    [wrapper_data] => Array
        (
            [0] => HTTP/1.1 200 OK
            [1] => Date: Mon, 04 Aug 2008 20:01:42 GMT
            (and so on...)
        )

[wrapper_type] => http
[stream_type] => tcp_socket/ssl
[mode] => r+
[unread_bytes] => 0
[seekable] => 
[uri] => https://www.example.com
[timed_out] => 
[blocked] => 1
[eof] => 
)

However, when I execute the same code on another server, i get this:

Array
(
    [wrapper_data] => Array
        (
            [headers] => Array
                (
                    [0] => HTTP/1.1 200 OK
                    [1] => Date: Mon, 04 Aug 2008 20:05:47 GMT
                    (and so on...)
                )

        [readbuf] => Resource id #3
    )

[wrapper_type] => cURL
[stream_type] => cURL
[mode] => r+
[unread_bytes] => 0
[seekable] => 
[uri] => https://www.example.com
[timed_out] => 
[blocked] => 1
[eof] => 1

I have to assume that the difference is that a different wrapper is being used in each case (evidienced by the different wrapper_type and stream_type properties). As a result, the latter contains a slightly more elaborate wrapper_data structure than the former.

While i don't think that normally this would be a problem, I believe one of the side effects is that cookie handling in an SSL envionment does not work on the second environment (which uses the cURL wrapper 😕); it works just fine on the on first enviroment (which uses the http wrapper 😕).

Short version is that i want the latter environment to produce results like in the former environment. I want to believe that the way to do this is to instruct PHP to use a different wrapper than what it uses by default, but i don't know how. I have scoured the php.net website to identify a method to do this, without sucess. I can't help but to think this would be defined in the INI file, but I'm not sure where.

Any thoughts on how i can cause PHP to use an alternative wrapper for https sockets?

[jazz_snob]

Oh man I've run into that: no curl module on the deployment environment. There are other options: curl from the command line for example.

Other than that, for sockets I've used perl or C. With php I got irradicate results reading the page content...it came in chunks and that really messed things up. But I don't think I used the stream function at the time (this was a few years ago).