Dr. Dave, author of the Spam Karma WordPress antispam plugin, has posted an interesting article about new weblog-spammer tactics:
These spams do not present most of the idiotic traits of their lower colleagues: they do not try cramming hundreds of URLs or inserting hundreds of easily spotted junk keywords in the comment content. Instead, they use only the dedicated name and homepage fields to sneak in spam URL and keywords. The comment content is often perfectly innocuous, sometimes even topical (by copying parts of another comment or a trackbacking post). All in all, these spams could easily be missed by a human moderator who wouldn’t look carefully at the contact name and URL.
(Thanks to Kelson Vibber for the pointer to this.)
In other words, he is noting what we noticed in email anti-spam; that what works well one year, is likely to degrade over time as the spammers attempt to evade it, and one has to keep working to keep up.
The best term for this appears to be adversarial classification. Anti-spam activities fall into this category, and it often means that classic text classification algorithms aren’t suitable — after all, the Reuters-21578 dataset never tried to evade your classifier ;)
In a similar vein, this MS research paper is interesting:
Previous work on adversarial classification has made the unrealistic assumption that the attacker has perfect knowledge of the classifier. …. We present efficient algorithms for reverse engineering linear classifiers with either continuous or Boolean features and demonstrate their effectiveness using real data from the domain of spam filtering.
It’s akin to John Graham-Cumming’s work looking into how a spammer could get past a bayesian filter “from the outside”, but with more techniques, and examining MS’ MaxEnt algorithm, too. PDF here, well worth a read.
(By the way, I’m in the process of moving house, so if you send me an email, it may take a while for me to reply. This situation is likely to prevail for the next few weeks, for what it’s worth — fun.)