Hi :>
i think it's better to dave a DB, but it will be a very big db file if your search engine automatically stores data of all the pages linked to each other. you have to find a way to select most important pages.
for example main pages of each website is more important , www.ourekbatan.com/index.htm is more important than www.ourekbatan.com/map/index.htm.
for this fact , I tried to count '/'s in URLs and URLs with less / are more important.
The other way is to find more dependable and important pages is this :
for example you have 10 keywords on one page and each of them had been repeated 10 times. on another page there is 2 keywords and they are repeated 50 times. I think the second page is more important.
so you can calculate the number of repeatation divided by number of keywords, and it will be a good and dependable thing to select the most important page.
I have to go now , Bye 🙂
Amin.
