Scaling Network Policy Enforcement Beyond the Cluster Boundary with Cilium
Talkat Cilium + eBPF Day North America 2024 · on Tuesday, 2024-11-12· at 13:30· Salt Palace | Level 1 | Grand Ballroom B·
summary
Datadog runs Cilium across hundreds of thousands of pods in hundreds of clusters and implements a custom dedicated etcd KV-store mesh that converts cluster mesh propagation from M×N → M+N API updates, dramatically lowering watch load. The change works in production after moving cluster metadata management into the operator and using a custom Helm chart. Attention to endpoint churn, KV store QPS tuning and the default lease TTL of 15 minutes is essential to avoid cross‑cluster policy enforcement gaps.
AI-written summary · about
chapters (15)
- 0:00Network policy enforcement intro
- 1:02Network policies in Kubernetes
- 3:31Cilium network policies
- 5:56Cilium identities and endpoints
- 7:20Scaling Cilium with CRDs
- 9:24Multi-cluster policy challenges
- 11:51Datadog chose Cilium Mesh
- 12:56Cluster Mesh basics
- 15:47Cluster Mesh API architecture
- 17:51KV store mesh improvements
- 19:15Datadog’s KV store mesh
- 21:04Large cluster migration lessons
- 22:49Stale keys and HA handling
- 23:51Monitoring Cluster Mesh
- 24:55Cluster Mesh future improvements
related talks
20mTackling Container Networking at Scale
32mBreaking Boundaries: Multi-cluster with Cilium cluster mesh
28mLessons from Building Scalable Network Policy Enforcement with eBPF
16mCilium Wars: Scaling Up Your Network with Jedi-like Cilium Handling Techniques - Yash Tandon
25mMore Churn No Problem: Lessons Learned Running Cilium in Production
31mCilium Mesh
35mTurn Your Kubernetes Cluster Into a Castle Using Advanced Cilium Network Policies - Philip Schmid
24mHigh-Scale Networking for ML Workloads With Cilium