Validating a URL

[D_tunisia]

Hi guys, wonder if you can help me out?

I have a script which checks if a URL exists:

function url_exists($strURL) {
    $resURL = curl_init();
    curl_setopt($resURL, CURLOPT_URL, $strURL);
    curl_setopt($resURL, CURLOPT_BINARYTRANSFER, 1);
    curl_setopt($resURL, CURLOPT_HEADERFUNCTION, 'curlHeaderCallback');
    curl_setopt($resURL, CURLOPT_FAILONERROR, 1);

curl_exec ($resURL);

$intReturnCode = curl_getinfo($resURL, CURLINFO_HTTP_CODE);
curl_close ($resURL);

if ($intReturnCode != 200 && $intReturnCode != 302 && $intReturnCode != 304) {
   return false;
}Else{
    return true ;
}
}

if(url_exists($url)) { // Check if the URL exists
echo "Good!"; }
ELSE { echo "Bad"; }

This works find with most pages, but with a few it gives an unusual result.

If I go to http://www.google.com it works fine.

If I go to http://www.nintendo.com or http://www.downloads.com I get a false result even though the page does exist.

If I go to http://nintendo.com or http://downloads.com I get a true result.

Can anyone help me out?

Thanks in advance!

[big_nerd]

Why don't you try seeing what code the websites are exporting?

In my test - I am getting HTTP/500 (Internal Server Error).

$url = "http://www.nintendo.com";

function url_exists($strURL,$debug = false) { 
    $resURL = curl_init(); 
    curl_setopt($resURL, CURLOPT_URL, $strURL); 
    curl_setopt($resURL, CURLOPT_BINARYTRANSFER, 1); 
    curl_setopt($resURL, CURLOPT_HEADERFUNCTION, 'curlHeaderCallback'); 
    curl_setopt($resURL, CURLOPT_FAILONERROR, 1); 

curl_exec ($resURL); 

$intReturnCode = curl_getinfo($resURL, CURLINFO_HTTP_CODE); 
curl_close ($resURL); 
if ($debug) {
	echo $intReturnCode."<br>";
}
if ($intReturnCode != 200 && $intReturnCode != 302 && $intReturnCode != 304) { 
   return true; 
} else{ 
    return true ; 
} 
} 

if(url_exists($url,true)) { // Check if the URL exists 
	echo "Good!"; } 
else { 
	echo "Bad"; 
}

Try using FireFox with Live HTTP Headers to view what the webpage(s) are trying to do.

Edit:

Here are the headers (first part):

http://www.nintendo.com/

GET / HTTP/1.1
Host: www.nintendo.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive

HTTP/1.x 302 Moved Temporarily
Set-Cookie: bigip_nintendow=2657182474.20480.0000; expires=Fri, 13-Jun-2008 15:57:16 GMT; path=/
Set-Cookie: NINTENDO=[my_ip_addy].1213369036314752; path=/; expires=Wed, 12-Jun-13 14:57:16 GMT; domain=.nintendo.com
Set-Cookie: JSESSIONID=[session id here]; Path=/
Date: Fri, 13 Jun 2008 14:57:16 GMT
Server: Apache
Location: http://www.nintendo.com/countryselector
Content-Length: 0
P3P: CP="CAO DSP COR CURa ADMa DEVa TAIa PSAa PSDa IVAa IVDa CONo OTPa OUR SAMa NOR IND PHY ONL UNI PUR COM NAV INT DEM CNT STA PRE OTC", policyref="http://www.nintendo.com/w3c/p3p.xml"
NODE: swebw8
Keep-Alive: timeout=120
Connection: Keep-Alive
Content-Type: httpd/unix-directory

It seems that the page is attempting to set Cookies, and redirect to a new page.