html parser

mcalpine

i need a script that can retrieve the title and all the anchor tags from an html page and put them into an array...

can anyone give me a sample script...

thanks..

superwormy

Heres the anchor tag script:


<?php  

// Function HTML HREF SPIDER

// array/FALSE HTML_HREF_Spider ( string string [ , string regex ] )
// returns FALSE if no matches found ( URL format DEFAULT ), array of matches on success


function HTML_HREF_Spider ($contents) {

$amatches = array ();
$matches = array ();
$alreadydone = array ();

while (stristr ($contents, "<a ")) {

	$contents = stristr ($contents, "<a ");
	$contents = substr ($contents, 3);

	$tostrip = stristr ($contents, "</a>");

	$amatches[] = str_replace ($tostrip, "", $contents);

}

foreach ($amatches as $key => $value) {

	$value = str_replace ("\n", " ", $value);
	$value = str_replace ("\r", " ", $value);

	$word = strstr ($value, ">");
	$word = substr ($word, 1);

	$href = stristr ($value, "href=\"");
	$href = substr ($href, 6);
	$tostrip = strstr ($href, "\"");
	$href = str_replace ($tostrip, "", $href);

	if (strstr ($word, "<") or strstr ($word, "/") or strstr ($href, "@")) {

		unset ($word);

	} else {

		$word = trim ($word);

		while (strstr ($word, "  ")) {

			$word = str_replace ("  ", " ", $word);

		}

	}

	if ($href) {

		if (!array_search ($href, $alreadydone)) {

			$href = trim ($href);

			$arrayname = "array_" . $counter;
			$$arrayname = array ($href, $word);

			$matches[] = $$arrayname;
			$alreadydone[] = $href;

		}

	}


}


if (count ($matches) > 0) {

	return ($matches);

} else {

	return (FALSE);

}


}



 ?>

I'll leave it up to tyou to find the title tag, try these links for a start:

www.php.net/stristr
www.php.net/str_replace
www.php.net/eregi_match
www.php.net/eregi
www.php.net/eregi_replace

mcalpine

im frustrated trying to make the anchor urls absolute urls and not relative....

does anyone have a script that will do this?

-m

superwormy

Sheesh do I have to do everything for you? ;-) Just kidding, here're some thoughts that might help, but no actual code, you should be able to write that yourself.

Take substr()'s ( www.php.net/substr ) or the first character, the first 6 characters, and the first seven characters. There are pretty much four ( aight, theres more, but I'll leave those up to you to find ) cases here we'll deal with.

[url]http://[/url] ( first seven chars )
[url]ftp://[/url] ( first six chars )
/ ( first one character )
[ anything else ]

If stristr ("http://", $first_seven_chars) {
it's already absolute, dont do anythign with it

If stristr ("ftp://", $first_six_chars) {
its an ftp site

If stristr ("/", $first_one_character) {
this is a url which just needs the domain name appended to the beginning

Else
Append the current path up to the last / to it.

$domainame = "http://mydomainname.com/path/to/file.html";
$temp = strrev ($domainname);

while (stristr ("/", $temp) {
$temp = stristr ("/", $temp);
$temp = substr ($temp);
}

$path_that_file_is_in = $temp;

Also:

www.php.net/parse_url

mcalpine

firstly, what with the name?

and thanks.....

i was having a problem with the relative thing because .... if you look at all the possible relative URLs ("../" , "../../," ) these can be infront ... and also what about get variables......
sometimes a get variable can be sent to a directory (using the start page of that DIR).... and they can be sent to a file directly...

see?

thanks...

-m

superwormy

Yeah, it really is a royal pain in the butt to do this. With the ../ things, just stristr ("/", $temp) once more everytime you find a ../

Also note that ./ would indicate the current directory, treat it the same as no slash at all.

Not sure what you mean about the GET thing... do you mean something like a URL like this:

/file/all/this/stuff/after/file/is/the/querystring

Where 'file' is the only actual file, and everything else is just query string?

Which has been force-typed with Apache? In a case like that theres nothing you can do... no way to tell what's a file and whats a directory.