Load Testing Kubernetes: How to Optimize Your Cluster Resource Allocation in Production
summary
Buffer uses load testing to pick per-pod resource limits so containers stop being killed and nodes pack more pods. Using loader.io for ramp and duration tests, metrics from cAdvisor/Heapster and a hacked dashboard called Kubb scope reveal when pods get OOMKilled, when CPU or memory is the bottleneck, and when response-time variance shows queuing. Memory went from 5M to 50M and CPU from 25m upward during tuning; this process lets Buffer move over half of production traffic to Kubernetes and tune etcd in a live demo.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 4:03What is optimal allocation
- 8:09Load testing strategies
- 8:50Coopscope
- 9:53Loader IO
- 16:30Loader IO Visualization
- 18:44Fail Logs
- 23:00How to Set Limits
- 25:10How to Know When Youve Gone Too Far
- 26:30What If Your App Is Inherently Performant
- 27:57Have We Automated This
- 30:05Could I Instrument This Locally
- 30:56Would I Run This in Production
- 31:50Do I Have a Complex Endpoint
- 32:25Conclusion
related talks
39mStaying in Tune: Optimize Kubernetes for Stability and Utilization
29mRight-Size Your Kubernetes Deployments
15mDemo: Resource(Cost) optimization on planet scale Kubernetes clusters.
36mDon't Be Greedy: Rightsize Your Kubernetes Cluster With Prometheus
05mResource Roulette: Winning the Kubernetes Allocation Game
38mDynamic Pod Resource Boundary Adjustment in Web Scale Clusters
32mOptimizing Application Performance on Kubernetes
51mWhy You Need Performance Tests for Proper Kubernetes Scaling