Continuous Optimization of MIcroservices using Machine Learning
summary
Twitter builds a Bayesian optimization service and uses it to tune a Java microservice by changing 30 JVM knobs simultaneously, running suggestions back-to-back against production traffic in staging. The system runs about ~80 probes and finds a configuration that yields roughly a 2x improvement in requests-per-second per GC cost. Implementation handles concurrent evaluations, experiment histories, constraint queries and pushes winning configs to the config service; hardware heterogeneity and noisy sensors require longer runs, isolation and early termination.
AI-written summary · about
chapters (26)
- 0:00Introduction
- 1:42The Problem
- 6:14Performance Stack
- 9:55Performance Tuning
- 13:22Performance Optimization
- 15:31Performance Constraints
- 17:44Hidden Variables
- 19:55Performance Tuning Problem
- 22:06Bayesian Optimization
- 23:48Example
- 24:58Gaussian Process
- 26:32Expected Improvement
- 29:54Bayesian Optimization as a Service
- 31:14Bayesian Optimization API
- 33:07Random Search
- 34:31Twitter
- 35:44Recap
- 36:28Microservice
- 36:53Staging
- 37:46Setup
- 40:18Results
- 42:19Optimization changes
- 44:17Takeaways
- 48:59Implementation
- 51:39Conclusion
- 53:51Question
related talks
29mContinuous tuning of Microservices via Bayesian Optimization
18mLet the machines optimize the machines
41mAutomated Performance Tuning with Bayesian Optimization
34mPerformance tuning Twitter services with Graal and Machine Learning
54mPerformance tuning Twitter services with Graal and Machine Learning
22mAutomating Performance Tuning with Machine Learning
49mPerformance tuning Twitter services with Graal and Machine Learning
56mPerformance tuning Twitter services with Graal and Machine Learning