Getting links

[Anon]

Hi,

I've read all through the regexp articles, but when I think I understand it, I try it out, and I can never get it working.

What i'm trying to do is:

I'm trying to strip out all links from a page, and return them to an array. I dont want the link text or anything.. just the actual link itself. PLUS, i dont want any links that to files/pages on another server. so any that start with http://, ftp, https..etc.. I dont want.

Can anyone help me with this? I assume i've got to use eregi(), but I tried this:

eregi("<a.* href=\"(.+)\">.+</a>", $contents, $dataArray);

and it didnt work.

Please excuse my ignorance.

[Anon]

Here is a function that you can send strings through and it will return an array of all the local links that the string contained. Query strings are preserved. Kinda alot of code, but it works. The problem that I saw that you would run into, is that html code doesn't have to follow strict rules. Therefore writing a regex could be difficult, because of the many variations in coding styles. For example, your regex assumes that the address is enclosed in double quotes. I have a habit of not putting my link addresses in double quotes. Yes it is sloppy, but it works.

function GetAddresses ($string){
$ct = 0;
$addressArray = array();
preg_match("/<a.<\/a>/",$string,$matchArray);
$matchArray = preg_replace("/\"/","",$matchArray);
$matchArray = preg_replace("/\'/","",$matchArray);
foreach($matchArray as $item){
$tmpArray = preg_split("/[\s]/",$item);
foreach($tmpArray as $item2){
if(preg_match("/^{href/",$item2)){}
list($stripped,$garbage) = split(">",$item2);
list($garbage,$address) = split("=",$stripped,2);
if(!preg_match("/^{http|ftp/",$address)){}
if(preg_match("/>/",$address)){
preg_match("/(.)>.*/",$address,$tmpArray2);
$address = $tmpArray2[1];
}
$addressArray[$ct] = $address;
$ct++;
}
}
}
}
return $addressArray;
}

[Anon]

Thanks a lot. I'll try that.

[Anon]

Hi,

I've tried that function, but it doesnt seem to be working on all pages..

http://www.bigbeigebox.net/tree.phps

thats the code i'm using. Am I doing anything obviously wrong?

[Anon]

Try this.

<?php

// functions
function GetAddresses ($string){
print "DEBUG - $string";
$ct = 0;
$addressArray = array();
$hmmArray = explode("\n",$string);
foreach ($hmmArray as $item){
preg_match("/<a.<\/a>/",$item,$matchArray);
$matchArray = preg_replace("/\"/","",$matchArray);
$matchArray = preg_replace("/\'/","",$matchArray);
foreach($matchArray as $item){
$tmpArray = preg_split("/[\s]/",$item);
foreach($tmpArray as $item2){
if(preg_match("/^{href/",$item2)){}
list($stripped,$garbage) = split(">",$item2);
list($garbage,$address) = split("=",$stripped,2);
if(!preg_match("/^{http|ftp/",$address)){}
if(preg_match("/>/",$address)){
preg_match("/(.)>.*/",$address,$tmpArray2);
$address = $tmpArray2[1];
}
$addressArray[$ct] = $address;
$ct++;
}
}
}
}
}
return $addressArray;
}

// error checking

if (!$url){
echo "Please enter a url";
exit;
}

// open connection & get page

$contents = "";

$fp = fsockopen ("$url", 80, $errno, $errstr, 30);

if (!$fp) {

echo "$errstr ($errno)<br />\n"; 

}else {

fputs($fp, "GET /$page HTTP/1.1\r\n"); 
fputs($fp, "Accept: */*\r\n"); 
fputs($fp, "Referer: hmm\r\n"); 
fputs($fp, "Accept-Language: en-gb\r\n"); 
fputs($fp, "User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)\r\n"); 
fputs($fp, "Host: $url\r\n\r\n"); 

while(!feof($fp)) { 
    $contents.= fgets($fp, 128); 
} 

fclose ($fp); 

// get title page 
$title = strip_tags($contents, '<title>'); 
eregi("(<title>)(.*)(</title>)",$title,$out); 
$title = $out[2]; 

// strip all other html tags 
$links = strip_tags($contents, '<a>');
$dataArray = GetAddresses($links);



// echo it all 
echo "<b>Page title:</b> $title<br />\n"; 
echo "<b>Number Of Links:</b> ".sizeof($dataArray)."<br /><br />\n"; 

for ($i=0; $i<sizeof($dataArray); $i++){ 
    echo $dataArray[$i]."<br />\n"; 
} 

}

?>

[Anon]

Remove the string with DEBUG - whoops.