Running Kubernetes on 'small scale' clusters - lessons learned
summary
Runs kOps on AWS with three self-hosted clusters, 35–40 nodes and ~300 pods each, and shows concrete configs for using kOps on AWS with 3 clusters, 35–40 nodes, ~300 pods each. Explains running everything on spot instances + node-termination-handler to cut costs, why multiple instance types and capacity-optimized allocation matter, and how cluster-autoscaler and py-kube-downscaler interact (and conflict) with ArgoCD sync windows. Demonstrates Goldilocks (VPA recommendation mode) for resource tuning and practical probe/startup settings that fixed JVM startup flapping.
AI-written summary · about
chapters (17)
- 0:00Introduction to small-scale Kubernetes
- 1:27Why this talk matters
- 3:12Cluster setup with KOPS
- 5:43Infrastructure on AWS
- 6:05Health checks and probes
- 8:32Managing secrets securely
- 9:37Deployment strategies
- 11:01Resource requests with Goldilocks
- 12:45Priority classes and QoS
- 14:34Cost optimization with spot instances
- 18:45On-demand base in KOPS
- 19:47Cost of test environments
- 21:12PR environment downscaling
- 24:21Solving downscaling conflicts
- 24:42Cluster autoscaler tuning
- 26:04Future plans and tools
- 27:06Q&A session
related talks
15mDemo: Resource(Cost) optimization on planet scale Kubernetes clusters.
47mPostmortem: A Modern Approach to Managing Kubernetes Clusters
44mFinOps at scale: Lessons from scaling Kubernetes workloads
36mOptimizing Resource Usage in Kubernetes
37mkubernetes on AWS gone wild
32mDon't Start With 500 Clusters: Patterns in Scaling Multi-Cluster Kubernetes…
25ma fully production Kubernetes cluster on AWS
21mLessons from the Field: Running Kubernetes at the Wellsite