curl linkback

[Chris-2k]

hi guys,

merry xmas!

Um, i have a working curl mod but iv just coded this 1 and need 2 know if it works:

        function subBacklink($link, $subBacklink) {
            global $subBacklink;

	$options = array(
		CURLOPT_RETURNTRANSFER => true,     // return web page
		CURLOPT_HEADER         => false,    // don't return headers
		CURLOPT_FOLLOWLOCATION => true,     // follow redirects
		CURLOPT_ENCODING       => "",       // handle all encodings
		CURLOPT_USERAGENT      => "spider", // who am i
		CURLOPT_AUTOREFERER    => true,     // set referer on redirect
		CURLOPT_CONNECTTIMEOUT => 10,      // timeout on connect
		CURLOPT_TIMEOUT        => 10,      // timeout on response
		CURLOPT_MAXREDIRS      => 1,       // stop after 10 redirects
	);

	$ch      = curl_init( $surl );
	curl_setopt_array( $ch, $options );
	$page = curl_exec( $ch );

	curl_close( $ch );

	}

can one say this will work ^......? Also i would like to add a

preg

check too.

Atm the 1 below works fine, but not for all sites, so im thinking teh curl above may work better......?

        function subBacklink($link, $subBacklink) {
            global $subBacklink;

        $ch = curl_init($link);  
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);  
//            curl_setopt($ch, CURLOPT_HEADER, FALSE);
              curl_setopt($ch, CURLOPT_USERAGENT, 'Opera/9.80 (Windows NT 5.1; U; en) Presto/2.7.62 Version/11.01');  

            $page = curl_exec($ch);  

            curl_close($ch); 
}

atm this is how im checking if a linkback is visble or not:

        if (stripos($page,$subBacklink) || stripos($page, str_replace("http://www.", "http://", $subBacklink)))
            return true;
            else 
            return false;
		}

so if theres no visible link, it does a blacklist of $surl otheerwise if there is a lnik it continues the script:

		if(subBacklink ($surl, $subBacklink) == false) {
            mysql_query("INSERT INTO wcddl_blacklist (url,reason,dat,email) VALUES ('".mysql_real_escape_string($surl)."','".mysql_real_escape_string($reason)."','".mysql_real_escape_string($date)."','".mysql_real_escape_string($email)."')");
            blacklistemail($email,$surl);
        }

so i need to know is my modded code working? and how to aadd a preg check........?

[sfullman]

this would get that sub back link assuming it was just a domain.name.com

$page= // .. text from the page
if(preg_match('/http\s*:\/\/([-a-z0-9]+\.)*'.str_replace('.','\.',$subBackLink).'/i',$page)){
   //the link is there
}

notice I added ([-a-z0-9]+.)* as a way to cover not just www but also any other cname of the domain like admin.domain.com - modify as necessary.

If the back link needs to be more complex with query string parameters, then you need to make your preg more complex.

what if they have the domain hidden though? or if it's commented out? I would strip all <!-- --> comments, and furthermore search for a full LINK i.e.

/<a[^{>]+href="https:\/\/} ... etc .. "[^>]>(.|\s)*?<\/a>/i

regular expressions usually take some thought to be robust - and testing

[Weedpacket]

sfullman wrote:

regular expressions usually take some thought to be robust - and testing

Indeed; in this case at least it would be better going with a pre-existing HTML parser, like the one built into the [man]DOM[/man] extension.

Then again, in this situation even that may not be enough, depending on whether "hidden" could mean "styled with display:none" or "white on a white background".

[sfullman]

i am "agreed with weed" except I don't have any experience with the DOM extension. it certainly doesn't hurt to develop and test regex expressions - trust me once you learn how to use them, your coding and error checking will be much more solid. Just be aware that on really huge texts you can sometimes be a bit slower.

[Chris-2k]

Whey! i did a preg and it works...

now how can i say:

if ($surl == blah.co.cc or blah.tk) {
   // do this
} else if ($surl   == blah.blogspot.com) {
   // do this 
}

then

[sfullman]

go to php.net/preg_match_all

this would put all the url's in an array which you can then loop through and compare to an array of url's that you have in your database for example

I will forego an example because preg_match_all in php.net has a very good example

[Chris-2k]

wots going wwrong here

		function checkdom($surl) {
            // Soft2050
            $freeTLDs = preg_match('@^(?:http:\/\/)?([^/]+)\.(co\.cc|tk)@i', $surl);
            $blogspots = preg_match('@^(?:http:\/\/)?([^/]+)\.(blogspot\.com)@i', $surl);

		if (isset($surl) && $surl = $freeTLDs) $reason = $reason2; // Give Rwaso2 

		else if (isset($surl) && $surl = $blogspots) $reason = $reason3; // Give Reason3

		return $reason;
	}

	if (checkdom($surl) == true) {
		echo $reason;
//            mysql_query("INSERT INTO wcddl_blacklist (url,reason,dat,email) VALUES ('".mysql_real_escape_string($surl)."','".mysql_real_escape_string($reason)."','".mysql_real_escape_string($date)."','".mysql_real_escape_string($email)."')");
        }  

1

[bradgrafelman]

Inside here:

        if (checkdom($surl) == true) {
            echo $reason;
//            mysql_query("INSERT INTO wcddl_blacklist (url,reason,dat,email) VALUES ('".mysql_real_escape_string($surl)."','".mysql_real_escape_string($reason)."','".mysql_real_escape_string($date)."','".mysql_real_escape_string($email)."')");
        } 

$reason is undefined.

EDIT: Similar issues inside your function as well; both $reason2 and $reason3 are undefined.

Do you have display_errors set to On and error_reporting set to E_ALL? If not, why not? If so, are you sure PHP isn't generating any error messages?