"Small Data" Machine Learning
summary
A program sanitizes a single-letter account’s reply stream by training on 8,500 labeled replies, extracting features like language, mention counts and punctuation, and classifying with logistic regression with synthetic oversampling to correct extreme class imbalance. It uses MongoDB + Twitter streaming (firehose) for data collection, blocks classified users via the Twitter API, and validates results with scikit‑learn; outcome is roughly 80% effective and noticeably cleaner but limited by API quirks and PHP’s math tooling.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 1:01Twitter spam problem
- 2:57Developing the cleaner tool
- 4:41Machine learning basics
- 7:05Algorithms and training
- 10:33Logistic regression
- 14:01Data collection and features
- 20:12Addressing dataset bias
- 22:52Training the model
- 30:10Code implementation
- 33:40Putting it all together
- 38:02Lessons and future work
- 41:55Q&A and conclusion
related talks
48mLearning Machine Learning
13mUsing JavaScript to Teach Machines Basic Shit Things
42mLearning Machine Learning
30mMachine Learning: Beyond the Tutorial
35mIntelligent Ruby: Getting Started with Machine Learning- 42mLearning Machine Learning
37mLearning Machine Learning
46mPractical Machine Learning and Rails