Subsetting in gRPC: How We Implemented It at Datadog
summary
Datadog replaces full round‑robin with randomized subsetting plus rendezvous hashing to cut connection overhead while keeping stickiness, using subset size = 10 by default. Server CPU load reports are smoothed with a 1 minute moving average and fed into a distributed PID loop that replaces WRR’s static cost function; PID drives per‑server weights applied every weight update = 1s. Result: far fewer connections and markedly reduced server‑side imbalance; random subsetting RFC merged, PID RFC under review.
AI-written summary · about
chapters (22)
- 0:00gRPC load balancing intro
- 1:46Round robin challenges
- 3:10Subsetting design goals
- 4:33Rendezvous hashing
- 6:19Random subsetting impact
- 7:46Rejecting deterministic subsetting
- 8:28WRR imbalance limits
- 10:13PID controller for balance
- 13:25Smoothing load reports
- 14:53Sync PID and WRR updates
- 15:56Tuning PID parameters
- 16:39Subsetting results
- 17:42Failure detection use case
- 18:43gRPC community PR
- 19:37Q&A: Rockset vs PID
- 20:38Q&A: Monetary and resource benefits
- 22:04Q&A: PID tuning
- 24:13Q&A: Community adoption
- 25:36Q&A: Connection churn
- 26:17Q&A: PID deployment cost
- 28:26Q&A: PID-WRR sync risks
- 30:12Q&A: Convergence speed
related talks
21mR2P2: Making RPCs first-class datacenter citizens
47mMonolith to Microserv. at 20M+ requests per second: Latency and Scale Challenges
58mSRE Your gRPC...
35mgRPC Load Balancing on Kubernetes - Jan Tattermusch, Google
26mFeature Work In GRPC; XDS And Not
16mTowards Optimal Rack-scale μs-level CPU Scheduling through In-Network Workload...
32mgRPC Deep Dive: Prevent Your Service From Overtaking Itself
25mAvoiding Overload in Distributed Systems