Autothrottle: A Practical Bi-Level Approach to Resource Management for SLO-Targeted...
Talkat 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI '24) · 2024-04-16..04-18·
summary
A bi-level framework called Autothrottle pairs lightweight per-service controllers (Captain) with an application-level learner (Tower) that outputs throttle ratio targets. Tower runs a contextual bandit online, Captain uses closed-loop control every second, and the system maps throttle ratio to end-to-end latency. Evaluated on three microservice apps with production traces and a 21-day replay, it reduces CPU use up to 26.21% versus the best baseline (and up to 93.84% versus all baselines), avoids many k8s violations (71 of 480 hours), and runs with low overhead.
AI-written summary · about
chapters (7)
- 0:00The microservice challenge
- 3:38Limitations of current methods
- 6:39The auto-throttle framework
- 8:11Using throttle ratios
- 10:44Controller implementations
- 12:54Evaluation and results
- 16:18Conclusion
related talks
14mPerformance Optimization in High-Volume Transaction Systems
14mSlowpoke: End-to-end Throughput Optimization Modeling for Microservice Applications
29mContinuous tuning of Microservices via Bayesian Optimization
29mCustom App Autoscaler Using Cloud Controller API - Christopher Decelles
28mOptimizing Istio Autoscaling: From Resource-Centric To Connection-Aware
30mThinner Clouds with Preallocation
11mLightning Talk: Grasp the Real Worth of Your Microservices by Brewing a Truth Poti...
32mTaking Care of Your Control Plane With API Priority and Fairness and Resource Quotas