How to create the best search engine

ceh7964

I have a large database holds information about books. ex: title, author, isbn, summary, ...

I need to tighten my search engine so the desired books can be located from my website. The site uses PHP and MySQL. From what I have found the first thing I need to do is index the proper fields in my table, using a FULLTEXT index. That, as far as I can tell is the best thing to do on the database side.

Then I need to widen my search results incase of miss spellings and the only thing I have found for that is the metaphone( ) function. Does anyone know if I am on the right track. The more I read about this metapone() function the more I am not sure.

Any advise is greatly appriciated

rachel2004

Be sure to also check out MySQL's MATCH and AGAINST for use with your FULLTEXT indexes:

http://dev.mysql.com/doc/mysql/en/Fulltext_Search.html

You might already be aware of it since it sounds like you've done some research on your own. Good for you, that's the way to go. ;-)

ceh7964

Thanks rachel2004 for the reply

I pretty much know about the MySQL index technique using the Match and against with FULLTEXT, but my main question is what can I use along with this to sharpen my searches.

The metaphone() function in PHP interest me the most because it seemed that it would solve the problem of misspelled search words, but the more I read I don't know if it would work because this function generates a metaphone key which, from my understanding has to be compared to another metaphone key for similarity. :bemused: So from my understanding I would have to pass every word in my table along with the search word to the metaphone() function, then compare the search word's metaphone key with every other word's metaphone key to find matching similarities. This would take for every with a large table.

Any suggestions about this or another way to work with miss spelled search terms.

Thanks a lot for any help

BuzzLY

I have researched this a little bit, and it seems that metaphone matching is really only good for matching up with keywords. If feasible, I would recommend creating a keyword field in your database, and as records are inserted, use PHP's metaphone() function to convert the keywords prior to insert.

I don't think metaphone will do what you are looking to do. If you think about it, though, metaphone really isn't good for doing initial searches of indexed text. It would return WAY too many results. It is better suited for things like spell-checking, or searching for a name/place.

For most searching, users want documents that give them an exact match. I would recommend giving them the exact match, and a link on the results page that allows them to search for similar sounding words.

The bottom line is, metaphone is really for use with a database built with metaphone searches in mind, not for use with existing databases.