Weird cURL issues

Lelend

Im trying to run a simple curl script that just fetches a plain old website. Nothing fancy. Here is the script I am using:

$ch = curl_init('http://www.google.com');

            curl_setopt($ch, CURLOPT_TIMEOUT, 30);
            curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
            curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

            $html_data = curl_exec($ch);

            curl_close($ch);
if (empty($html_data)) {
        print "It broke!";
}
else {
        print $html_data;
}

When I view the page I get the generic "The page cannot be displayed" error. But heres the funnything, when I run curl from the command line it fetches the entire site. Something funky is going on with curl and I cant quite put my finger on it. Was hoping to get a little help here.

Lelend

No one has any idea what the problem could be? I can open pages using fsockopen as well, just seems that curl is acting funny. Here is my php/curl information.

libcurl/7.10.6 OpenSSL/0.9.7a ipv6 zlib/1.1.4

Bobulous

Have you configured PHP to use the CURL extension?

I downloaded the libcurl executable, then discovered that PHP doesn't use it. It uses its own extension, which is included with the PHP download. You just need to edit your php.ini file to turn it on.

If you've already done that and checked you didn't miss a step, then I don't know why it'd misbehave in PHP.

Lelend

Actually I finally figured it out using a strace on the httpd process. I have to use a high file descriptor limit on my servers due to the amount of sites they host. Because of this I have to rebuild many rpms, including httpd and php. Apparently curl is one of those rpms I have to rebuild now after I raise theFD_SET_SIZE value. But thank you very much for the help =)

MarkR

You should not change FD_SET_SIZE !!

Just use a web server which doesn't use select() in the first place. Does Apache even use select() ?

Even without a changed FD_SET_SIZE, processes which use a lot of file descriptors can still monitor them all, provided they use poll() or epoll() instead. These interfaces are also a lot more efficient than select()

Mark