Highly Scalable Discriminative Spam Filtering
summary
Claims that global volume is huge — about 3.2 billion email accounts with 70% of all emails as spam and >10% undetected — and shows concrete pipelines: character n‑grams, avoid stemming, prefer binary bag‑of‑words over term frequencies, normalize lengths, apply the hashing trick to collapse millions of attributes. Uses convex discriminative learners (SVM/logistic regression) with L2 regularization, trained by stochastic gradient descent and parallel model‑averaging or consensus propagation to scale to hundreds of millions of instances. Recommendations: keep all labeled data, skip explicit feature selection, retrain iteratively.
AI-written summary · about
chapters (7)
- 0:00Introduction to spam filtering
- 1:34Problem setting and goals
- 6:18Data provisioning challenges
- 8:29Feature extraction techniques
- 14:31Large-scale learning algorithms
- 21:53Key recommendations summary
- 23:16Q&A session
related talks
42mBuilding a Robust Fraud Detection System
23mFighting phishing and spam with online machine learning on data streams
16mProduction MLOps for Spam Detection: Real-Time Multi-Modal AI
27mIs that spam in my ham? A novice's inquiry into classification.
42mDoes Spam have an opinion?
37mAutomated Machine Learning: Building a Pipeline
41mChallenges of Machine Learning at Scale
21mFinding Spam in Short Text Fields