Cannot use curl to access some sites

teguh123

<?
function usecurl1($url)
{
$ch = curl_init();
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_URL, $url );
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/1.6");

$output['html'] = curl_exec ( $ch );
$output['opt'] = curl_getinfo ($ch);
curl_close($ch);

    return $output;   
}
function usecurl($url)
{
    $output=usecurl1($url);
    return $output['html'];
}

$output=usecurl1("http://en.wikipedia.org/wiki/Special:Random"); //doesn't work
print_r ($output);

echo '<br>';

$output=usecurl1("http://yahoo.com");    //works
print_r ($output);

Why?

bradgrafelman

If you'll notice in the return code for both URL's, the server is redirecting you to another location (301 = temporary move, 302 = permanent move, IIRC).

Thus, you need to use [man]curl_setopt/man to enable the CURLOPT_FOLLOWLOCATION option so that cURL follows all of the redirects to retrieve the actual page.

teguh123

I thought CURLOPT_FOLLOWLOCATION default to true?

bradgrafelman

I don't think that it does; when I added that to your code snippet, I got the HTML you're after for the Wikipedia link.

teguh123

It works. It turns out I updated a different file.

Thanks guys.

teguh123

Guys,

It works on my dedicated server. It doesn't work on my reseller account.

Some problem with safemode.

Warning: curl_setopt() [function.curl-setopt]: CURLOPT_FOLLOWLOCATION cannot be activated when in safe_mode or an open_basedir is set in /home/personal/public_html/tools.php on line 10
Array (

 => 1 [opt] => Array ( [url] => http://sr.wikipedia.org/wiki/%D0%9F%D0%BE%D1%81%D0%B5%D0%B1%D0%BD%D0%BE:Random [content_type] => text/html; charset=utf-8 [http_code] => 302 [header_size] => 756 [request_size] => 207 [filetime] => -1 [ssl_verify_result] => 0 [redirect_count] => 0 [total_time] => 0.154042 [namelookup_time] => 0.002607 [connect_time] => 0.029417 [pretransfer_time] => 0.02947 [size_upload] => 0 [size_download] => 0 [speed_download] => 0 [speed_upload] => 0 [download_content_length] => 0 [upload_content_length] => 0 [starttransfer_time] => 0.154008 [redirect_time] => 0 ) [url] => http://sr.wikipedia.org/wiki/%D0%9F%D0%BE%D1%81%D0%B5%D0%B1%D0%BD%D0%BE:Random [change] => yes ) 

Now, I thought I want to do curlopt_follow manually. Basically I would check whether [http_code]  is 301 or 302 and then redirect appropriately to the right URL.

The thing is I don't know what the right URL is. How should I know?

The thing is curl_getinfo  doesn't provide any information where the page is redirected too.