On Modular Learning of Distributed Systems for Predicting End-to-End Latency
summary
Fluxion builds a modular performance model by packaging per-service ML models into learning assignments and wiring them into an inference graph so changes only require reassembling affected assignments. Implementation uses Thompson sampling for input selection and differential evolution for output weighting, and is agnostic to specific ML algorithms. Evaluated on three microservice apps (up to 142 services on a 100-VM cluster), it cuts MAE by up to 68.41% versus monolithic models and speeds 90th-percentile latency tuning by up to 1.57×.
AI-written summary · about
chapters (10)
- 0:00Start
- 0:22Performance Prediction for Cloud Systems (2)
- 5:13Key Observations and Intuitions (2)
- 9:05Learning Assignment - Inputs
- 10:44Inference Graph
- 14:05Evaluation Setup
- 14:26Evaluation Methodology
- 15:55Perf Modeling Error Reduction - Train Ticket
- 17:35Application p90 latency speedup - Train Ticket
- 18:10Conclusion
related talks
14mPerformance Optimization in High-Volume Transaction Systems
42mML Latency No More- Useful Patterns to Reduce ML Prediction Latency to Sub X ms
11mScaling Distributed Machine Learning with In-Network Aggregation
47mImprove Load Balancing With Machine Learning Techniques Based on the Sched_ext Framework
21mHarnessing Microservices for Scalable, Fault-Tolerant ML Systems
57mMachine Learning in the Age of IoT
31mLeveraging GraphQL for Continual Learning in Real-Time ML Systems
26mPatterns of ML Models in Production