Load is not what you should balance: Introducing Prequal
Talkat 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI '24) · 2024-04-16..04-18·
summary
PReQuaL is a load balancer that actively probes servers and uses the power-of-d-choices with asynchronous reusable probes to pick based on estimated latency and requests-in-flight (RIF) rather than CPU. It keeps a probe pool (≈16 probes), typically sends 2–3 probes per query, and applies a hot/cold lexicographic selection to avoid the hottest 20% of machines. Deployed inside YouTube for >1 year, it cuts tail latency (99th reduced from ~2→1.2 normalized), lowers error rates, and saves peak RAM and CPU headroom.
AI-written summary · about
chapters (10)
- 0:00Introduction to Prequel
- 0:15Core balancing principles
- 5:03Selection logic
- 6:16Managing the probe pool
- 6:44Algorithm performance
- 7:55CPU load and latency
- 8:41Optimizing algorithm parameters
- 10:02Comparing balancing policies
- 14:22Competitive analysis
- 14:59Conclusion
related talks
41mLoad Balancing Roundup
46mKeeping the Balance: Load Balancing Demystified
17mQDSR: Accelerating Layer-7 Load Balancing by Direct Server Return with QUIC
13mEdgeBalance: Model-Based Load Balancing for Network Edge Data Planes
22mLoad Balancing in gRPC
30mKeeping the Balance: Internet-Scale Loadbalancing Demystified
35mSolving High-load Load Balancing Problems
25mLoad Balancing in gRPC