IPVS for Scaled Private Cloud Load Balancing [I]
summary
Comcast VIPER replaces external appliances with IPVS direct-reply inside Kubernetes and a controller called Kube2IPVS that stores VIP-to-service mappings in config maps and writes iptables chains to jump into kube-proxy service chains. The system preserves L2 adjacency, mitigates ARP races and master/backend co-location, switches iptables updates to save/restore to avoid locks, enables hairpin for same-node clients, removes nodePort conflicts, and claims sustained egress of over 500Gbit/s from a modest cluster while supporting dynamic VIP allocation.
AI-written summary · about
chapters (22)
- 0:00Introduction
- 0:38What is VIPER
- 1:17How and why we began to use Kubernetes
- 2:52What we needed
- 3:29The clusion
- 4:04What is IPVS
- 4:44Direct Reply
- 6:18Docker Run
- 7:05Why was this well
- 7:45Additional problems
- 9:18Implementing IPVS
- 10:08Cube to IBBs
- 10:45How IPVS Works
- 12:46Cube to IPVS
- 13:22Backend Service
- 13:53IPVS Master
- 14:25List of Nodes
- 15:15Config Map
- 16:55Integration Issues
- 18:16Colocation Issues
- 19:15IP Tables
- 20:08Demo
related talks
36mUnderstanding How kube-proxy Works in IPVS Mode
38mVPCs in Kubernetes
40mCloud, Kubernetes, and Service Networking - Taming the Turtles
21mVPC in k8s: Not As Simple As It Looks
40mCase Study: Kubernetes at Comcast
35mLiberating Kubernetes From Kube-proxy and Iptables
35mIntelligent Load Balancing in Kubernetes
29mCalico/VPP: Using Calico’s Pluggable Dataplanes for Fun and Fast Networking