Under Pressure: Applying ML in Real Time
summary
A real-time ML pipeline is built to hit 100 transactions/sec with <=100 ms latency on a single moderately powerful machine by iterating tools and architecture: initial batch PySpark ETL, LightGBM models tracked in MLflow, Kafka for streaming, Postgres and Redis for historical and current context, then replacing Faust with Java Flink and finally joining enrichment and inference so the model runs embedded in the Flink task. The result is a working prototype that removes the REST bottleneck and supports live retraining and state updates.
AI-written summary · about
chapters (6)
- 0:00Real-time ML systems intro
- 3:39Real-time ML scenarios
- 5:46RTML system architecture
- 11:24How not to build RTML
- 19:32Lessons from the journey
- 20:56Q&A
related talks
38mReal Time ML Pipelines in Multi-Tenant Environments
39mData Pipelining for Real-time ML Models
31mLeveraging GraphQL for Continual Learning in Real-Time ML Systems
40mReal-time Learning with Spark
32mLiveMLP: ML Platform for Assisting Contact Center Agents in Real-Time
37mBig Data Analytics, Machine Learning, Real Time Processing
30mReal-Time ML services: From ideation to production
14mPerformance Optimization in High-Volume Transaction Systems