Automated Performance Tuning with Bayesian Optimization
summary
Describes a production system that uses Bayesian optimization (Spearmint) to tune JVM and container knobs via Mesos/Aurora orchestration, injecting configs and running experiments on real instances. It targets ~30 JVM parameters, optimizes the metric requests/sec divided by GC cost, and finds near-optimal settings within ~20–25 iterations, doubled throughput in a proof‑of‑concept. The service called Auto-tune runs staged and in-production experiments, handles rollbacks, factors hardware heterogeneity, parallelizes trials, and feeds results back for continuous optimization with Spearmint.
AI-written summary · about
chapters (7)
- 0:00Performance at Twitter
- 6:09The tuning problem
- 11:39Bayesian optimization
- 18:55Tuning process example
- 23:41Proof of concept
- 28:21Implementing Auto-tune
- 37:26Conclusion and Q&A
related talks
29mContinuous tuning of Microservices via Bayesian Optimization
01h 01mContinuous Optimization of MIcroservices using Machine Learning
22mAutomating Performance Tuning with Machine Learning
18mLet the machines optimize the machines
46mLinux Kernel Autotuning
27mLeverage ML to automatically tune K8s apps
56mPerformance tuning Twitter services with Graal and Machine Learning
34mPerformance tuning Twitter services with Graal and Machine Learning