Need serious help with regex ...

Anon

Hello all.

I had the idea of creting my own PHP search engine, which is trying to deal with data outputed by google.

I found out the the only way to grep the data I want to use is by using regex ... oohhhh god =)

I need serious help with this. Below is a simple, one result search from google.

<a href=http://www.php.net/>PHP: Hypertext Preprocessor</a> ... Contact. Please submit website bugs in the bug system. You can contact the webmaster at webmaster@php.net. PHP Magazin out now! ... New PHP.net URL Howto. ...

What I wish to get from google is the URL, the Title + the Description. So it can be breaked down pretty much to this ;

<a href=$URL>$TITLE</a> $DESCRITPION

Easy ? Well, not really ... Sometimes, the HTML is formatted differently, because Google may add a 'translate' function, like this ;

<a href=URL>$TITLE - [ <a href=$TRANSLATE_URL... class=fl>Translate this page</a> ] $DESCRIPTION

Also, in some case, a given search result does not output neither a title, nor a description, so we get something like this ;

<a href=$URL>$TITLE</a> ...

I think those a the major cases of exceptions, but there a surelly more of them. I browsed a little over the many regex tutorials on the net, and this did not help much. It seams like im trying to do something that is way over my skills right now, and that is why im calling from help on this board =)

Thanks for any clue or advice.

van__

This will get you the link.
Is raw and not currently used by me, but works.

I keep it around for those who ask for clues.
<?
function parseLinks($strFile) {
/
parses out all <a hrefs> and creates GLOBALS['LINK_ARRAY']
ARRAY is structured as follows:
$GLOBALS['LINK_ARRAY'][0][$key] contains the full HTML link
$GLOBALS['LINK_ARRAY'][1][$key] contains the actual link URL
$GLOBALS['LINK_ARRAY'][2][$key] contains the linking page text
/
@preg_match_all('/<A+ [?]|HREF=+\'\"(.[^{>]+)\'\">(.[^{<]+)<\/A>/i',}}
$strFile, $GLOBALS['LINK_ARRAY']);
}

parseLinks($strFile);
print_r($GLOBALS['LINK_ARRAY']);
?>

A full page parser that has some AI and STRONG pattern matching is what you need here, since word association and limited intelligence is required to intelligently traverse the world wide web...

So you probably are in over your head.

Maybe you should check out http://htdig.org.

Word of advice: Make sure you have LOTS of storage, spiders are data whores.