Machine Learning is Going Real-Time
summary
Real-time ML splits into Level 1: online predictions (ms–1s latency) and Level 2: online learning (minutes, micro-batches of ~100–500); TikTok uses real-time updates to learn preferences, improving recommendations. Latency matters: raising query latency from 100→400 ms cuts searches ~0.6% and other studies show conversion drops up to 30%. Production techniques include model compression/quantization and distillation, hardware accelerators, streaming stacks like Kafka + Flink, and examples: Roblox reports ~30× latency reductions; Weibo runs ~10‑minute online update cycles.
AI-written summary · about
chapters (9)
- 0:00Introduction and background
- 1:49Real-time ML definitions
- 3:13Online predictions importance
- 8:29Fast inference techniques
- 11:18Real-time pipelines
- 16:31Request vs event driven
- 20:25Online learning basics
- 22:10Use cases for online learning
- 25:17Q&A and closing
related talks
19mReal-Time AI: Low Latency Solutions for Interactive Applications
24mUnder Pressure: Applying ML in Real Time
40mReal-time Learning with Spark
30mReal-Time ML services: From ideation to production
53mMachine Learning Platforms
30mReconceptualizing Machine Learning for the Real-time World
42mML Latency No More- Useful Patterns to Reduce ML Prediction Latency to Sub X ms
29mMachine Learning in Production: What to do after you’ve trained the models?