Switching your CNI — Simone Sciarrati & Federico Hernandez
summary
40+ teams, 3,000+ deployments and ~10,000 pods move from the AWS VPC CNI to Cilium without creating new clusters by performing an in-place live migration over a few days, automating node labeling and drains with scripts and Helm-managed Cilium. Tests use production-scale load generators; problems include daemonset resource pressure, GPU node IP-limit constraints, and a Cilium routing bug triggered by heavy IP reuse that required a PR to fix. Final setup runs Cilium proxyless while using AWS native encryption for node traffic.
AI-written summary · about
chapters (16)
- 0:00Introduction and speakers
- 1:24Meltwater engineering platform
- 3:54Why switch from AWS CNI
- 6:01Migration requirements
- 8:12Cilium setup with COPS
- 10:02Performance testing Cilium
- 10:24Live migration strategy
- 14:02Automating the migration
- 15:58First production migration
- 18:24Post-migration DNS issues
- 21:56Debugging the DNS problem
- 23:40Bug fix and lessons learned
- 24:45Cilium proxy-less mode
- 25:28Future cluster upgrades
- 26:31Case study and resources
- 27:12Q&A session
related talks
26mThe Road To Cilium: Migrating 150+ Kubernetes Clusters at Airbnb
28mLive Migrating Production Clusters From Calico to Cilium
25mMore Churn No Problem: Lessons Learned Running Cilium in Production
24mConfluent's Multi-Cloud Journey to Cilium: Pitfalls and Lessons Learned
20mTackling Container Networking at Scale
24mHigh-Scale Networking for ML Workloads With Cilium
28mA Decade of Cilium Around the World
28mConnecting the World: Your Hands-on Guide To Cilium ClusterMesh and MCS-API