Scaling Kubernetes Application to 1M Users
summary
Scaling strategy centers on concrete limits and tools: pods scale with HPA but can take up to 1.5 minutes to react, while node provisioning via Cluster Autoscaler can take 4–5 minutes; replacing those with KEDA + Carpenter moves pod and node scaling to seconds and supports idle-zero cold starts. Control plane default concurrency in managed EKS sums to 600 (400 read / 200 mutating) so APF math yields ~1,187 concurrent requests for a 120-share workload. Add node-local DNS cache, custom networking/prefix delegation, and OpenTelemetry/Grafana monitoring to observe and avoid these bottlenecks.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 2:52Scaling with HPA and KEDA
- 7:24Scaling infrastructure with autoscaler
- 10:12Control plane tuning
- 16:32CoreDNS optimization
- 22:29Monitoring for scaling
- 26:47Key takeaways
- 28:23Multi-cluster scaling
related talks
32mScaling up to your first 10 million users - The Kubernetes Edition
28mScaling in Kubernetes Safely on On-Prem KaaS Across 1,300+ Clusters and 40,000+ Nodes
38mScaling EDA Workloads with Kubernetes, KEDA & Karpenter
46mScaling EDA Workloads with Kubernetes, KEDA & Karpenter
34mPanel: Improving and Managing Kubernetes at Scale
34mScaling Kubernetes up to 1,000 Control Planes - Dario Tranchitella
08mGetting Creative with HPA Scaling in Kubernetes
42mScaling Kubernetes-based Event-driven Workloads with Keda & Karpenter