If you want to run your own search engine on a subsection of the internet, I suggest you take a look at Mnogosearch (http://search.mnogo.ru). In brief: a search engine exists of three parts: The spider(which collect the url's you want in your search engine), the indexer(which grabs the content of those pages) and the search engine(containing the search algoritms). Mnogo provides the spider and the indexer, the data is stored in a mysql database. There is also a PHP frontend(for handling the search algorithm) available. I have extended this php frontend with search capabilities for synonyms and a advanced categorisation tool. You can see the result at www.com4ski.com. MOST INTERESTING OF ALL: everything is free for UNIX platform so definetly take a look at it.