Good, Bad, and Ugly - the Art of Load Balancing at Scale
summary
Claims concrete tradeoffs and a proven recipe: Cloudflare fronting AWS Network Load Balancers into multiple single‑zone Kubernetes clusters, 100% on spot instances, nginx as a DaemonSet to avoid 5‑minute NLB registration delays, Kubernetes events plus three‑probe health checks, and leaky‑bucket buffering. Shows real failures: a canary with one pod causes 100% 404s, weighted round robin and least‑conn pitfalls, and a production fleet of 6,000 containers across ~100 microservices at 100–200 external RPS.
AI-written summary · about
chapters (15)
- 0:00Load balancing at scale
- 2:18Static to dynamic balancing
- 3:21Round robin algorithm limits
- 5:47Weighted round robin for canary
- 7:13Canary releases risks
- 9:53Least connections pitfalls
- 12:57Service discovery at scale
- 16:35Kubernetes health checks impact
- 20:31Graceful shutdown safety
- 24:46Boring tech over hype
- 25:49Caching strategies trade-offs
- 29:17Leaky bucket traffic control
- 31:01Caching placement CDN benefits
- 31:44Production showcase cost optimization
- 36:21Q&A
related talks
18mDeveloping a Custom Load Balancer to serve traffic at scale
30mKeeping the Balance: Internet-Scale Loadbalancing Demystified
46mKeeping the Balance: Load Balancing Demystified
41mKubernetes the Very Hard Way
33mHow We Achieved Even Load Balancing on the Frontend Cluster
42mFrom Localhost to Production
29mLoad balancing: an in-depth study to scale @ 80K TPS
35mMake Load-Balancing Great Again with !