Weedpacket discriminator got very good at distinguishing them because it learned the rule "odd-numbered messages are ham, even-numbered messages are spam".
This is really funny. I feel like generative AI is probably absolutely full of problems like this, but everyone seems hell bent on having it generate all our content and pedagogical materials now.
EDIT: regarding sequence of training input: part of the algorithm involves randomly shuffling your data corpus.
Weedpacket I'm wondering about the vocabulary. At what point does this become part of the network?
I'm not sure what you mean by this. Broadly speaking, I can say that your vocabulary should be specific to your application. I.e., it would not be effective to use a vocabulary derived from spanish messages to train your machine on messages written in English. I imagine this website's vocabulary would be different than that for a forum devoted to old Throbbing Gristle records.
EDIT: I think by 'network' you mean 'Artificial Neural Network' here. Firstly, I believe I implemented a Support Vector Machine and got similar results to a FANN. The vocabulary is generated first -- you do a sort of pre-munge of all the messages to try and identify commonly used words. It is not helpful to examine words that only appear in a single message -- or no messages at all. The class instructed us that any word in the vocabulary should probably appear in at least 10-15% of the messages in your overall corpus to be useful. Once you have the vocabulary, you train the SVM/FANN by generating an array of features for each message. This feature array indicates for each word in your vocabulary whether that word is present or not in the current message. Any currency expressions are converted to some string like EXPRCURRENCY and urls to EXPRHTTPLINK or something like that so the absence or presence of currency expressions and links are examined. The SVM/FANN ends up with some array that assigns a weight to each word in the vocabulary (plus one extra baseline constant). A new message gets its own feature array by comparing it to the vocabulary, then you use your weighted SVM/FANN array to perform the ham/spam classification.
Weedpacket how does it help the network identify spammy features that aren't part of the vocabulary?
It doesn't! I've provided the description above expressly to point out what this filter does. I don't think it would be hard to expand beyond a word vocabulary to other features (this is a technical term used in the class to describe the input variables). I can think of a couple of other types of features:
- a series of binary indicators regarding the language (or charset) in which the message is written. e.g., isChinese, isEnglish, isSpanish, isRussian. I expect isChinese values of 1/true would be a strong indicator that a message is spam.
- a series of binary indicators that the message contains one or more links to a particular TLD. E.g., hasCOM, hasNET, hasUK, hasRU, hasCN, hasBIZ, hasINFO, hasTV, etc.
NOTE that the feature arrays we generate while munging messages don't use these associative keys I'm suggesting here. I just use the associative key names to try and suggest what's in a feature array. When the heavy lifting gets done, feature arrays are numerically indexed arrays with the exact same number of features (hundreds or thousands of them). The training phase involves a huge matrix multiplication.
I welcome any suggestions about other features.
Weedpacket Wouldn't the network itself develop triggers for certain sequences of characters (and correlations between the occurrences of such sequences) that are more predominantly spam than ham?
The particular algorithm I've implemented for this is an example from class, and probably quite simple compared to some of the more advanced possibilities out there today. I didn't bother to implement it on my personal website's contact form because I deployed recaptcha V3 and the spam stopped instantly. Disconcertingly, I get no correspondence at all now, and I wonder if any human being has tried to send me some actual ham.
I will note that this type of classifier is offered by Amazon as a Binary Classification Model and they charge pretty stiff rates for this sort of simple classifier. I'm not sure what, if anything, might make their classifier fancier than my algorithm here. Anecdotally, I experimented with one of these binary classifiers for a day or two and then shut it down using the AWS console. Something went wrong, however, and the classifier was not terminated and I got a bill several weeks later for over $200USD. I complained and Amazon refunded the money, but this sort of capability can be fairly flexible and the tech cos are charging a fair penny for it.