Pods on Steroids - A Case Study of Migrating Flipkart.com to K8s on Bare Metal
Talkat ContainerCon @ Open Source Summit Europe 2022 · on Friday, 2022-09-16· at 14:50· Wicklow Hall 1 (Level 2), Convention Centre Dublin, Spencer Dock, N Wall Quay, North Wall Dublin 1, D01 T1·
summary
Flipkart migrates from a QEMU/KVM VM stack to Kubernetes on bare metal, replacing vhost‑net with SR‑IOV and Cilium, running clusters that span 350k cores, 30k pods, 3k services. Benchmarks show an 80% throughput increase vs VM vhost‑net and ~45–60% mean/tail latency improvements; Mappy gains 40% throughput, Athena sees ~60% latency and throughput wins. Replacing VM control plane and daemon duplication saves ~4 cores/node (~40k cores), with projected total savings around 140k cores. Cluster API, Tinkerbell and DB operators are in progress and >90% stateless services run on K8s.
AI-written summary · about
chapters (10)
- 0:00Start
- 0:28Agenda
- 2:45Auto Scaling
- 5:23Why Did We Choose To Host Small Number of Large Clusters versus a Large Number of Small Clusters
- 11:26The Considerations for Running Kubernetes on Bare Metal
- 13:22Networking Setup for Virtual Machines
- 14:26Run Kubernetes on Vms Instead of Bare Metal
- 17:45Hundred Percent Read Test
- 19:47Real World Benchmarks
- 20:53Athena
related talks
32mMigrating 2 million CPU cores to Kubernetes
50mRunning Databases Efficiently at Scale - Case Study of Flipka...
29mFlipkart: Scaling Aerospike at Flipkart with Kubernetes
24mMigrating 2 million CPU cores to Kubernetes
32mRunning eBay’s High-Performance Workloads with Kubernetes
15mFrom ESXi to Kubernetes: Modernizing 1,400 Edge Locations with Open Source
29mFrom EC2 to Kubernetes Takeaways from a Brownfield Migration
24mOptimizing K8s and unlock blockers with Pod Live Migration