[RESOLVED] matching against list of stopwords

schwim

Hi there folks!

I'm working on a "similar links" function and plan on first stripping the common search stop words out of the string and then using similar_text to find the most similar links in the database. I'm doing fine on the tests, but currently am using a prepared array for the stop words and would instead like to modify the check to utilize a list of words contained in an easily managed text file.

Currently, I'm testing like this:

$string = 'After sustaining a life changing injury in 2013, Martyn Ashton, mountain bike and Road Bike Party legend, is Back On Track.';
$stopwords = array("able", "about", "above", "abroad", "according", "accordingly", "across", "actually", "adj", "after", "afterwards", "again", "against");

foreach ($stopwords as &$word) {
    $word = '/\b' . preg_quote($word, '/') . '\b/';
}
$useful_words = preg_replace($stopwords, '', $string);

And I would like to instead do the match incorporating a file with just a list of words each separated by a new line so I can easily modify the list without having to deal with quotes and commas:

[ in the file ]
able
about
above
abroad
according
accordingly
across
actually
adj
after
afterwards
again
against

Could someone help me with how I might achieve this? Thanks for your time!

Derokorian

take a look at [man]file[/man]

schwim

Edited for progress

Thanks a bunch for your help!

I got the array built but am now having a problem strippping the words from a string. I've tried:

(This was used in an earlier version of my attempts and worked then but now neglects to remove any stop words)

$clean_string = preg_replace('/\b('.implode('|',$stopwords).')\b/','',$input);

(This was from the example found on the web showing me how to strip words from a string. It gives me an error of preg_replace(): Delimiter must not be alphanumeric or backslash)

$clean_string = preg_replace("\b$stopwords\b", "", $input);

So far, I've been unable to find a working method of removing the stop words from a string. Any help would be greatly appreciated!

schwim

AHA! "FILE_IGNORE_NEW_LINES" saves the day!

dalecosp

schwim;11050629 wrote:
AHA! "FILE_IGNORE_NEW_LINES" saves the day!

If I had 100 bucks for every time data encoding caused a bug I'd be rich beyond the dreams of avarice! :p

Glad you found it 🙂