I know that this is not a strictly php question but most of you work with mysql so I thought that I would throw it out there and see if anyone knows how to do this. I am creating a Knowledge Base and in this Knowledge base users can attach files to their articles. I am then taking the files and extracting the text from them and saving it into the database so that it will be indexed for doing full-text searches against. My question is, is there a way to remove the stopwords to optimize the text so that it takes up less space in the database. I cannot just remove the stopwords because exact phrase searches do not work for example if I had a file that contained “problems with windows xp“ and I removed the stopwords so the text was saved as “problems windows xp“ and then try to do a search against the index for the exact phrase “problems with windows xp“ it will not find it. Any ideas? Or is there a better way to index the text from these files?
