Efficient load balancing in distributed systems
summary
Compares simple policies (round‑robin, random, least‑connections) with a simulated join-the-shortest-queue (2‑choice) that models request latency as log-normal latency and analyzes the balls‑into‑bins imbalance: random runs show max ~139 vs min ~100 (SD ~12), latency skew raises variance, while JSQ cuts SD to ~3 and max/min to ~127/114. Covers ADCs and tools (hardware F5, software Nginx, Node.js seaport), shows a small Node.js simulator and argues JSQ yields much tighter balancing in practice.
AI-written summary · about
chapters (25)
- 0:00Introduction
- 1:19About me
- 2:04What is load balancing
- 2:18Optimal load balancing
- 3:06Simplification
- 4:45Failover
- 6:09ADC
- 7:59Hardwarebased
- 8:23Softwarebased
- 8:46Algorithms
- 9:42Roundrobin
- 10:03Cons
- 11:14Edge cases
- 12:32Balls into bins
- 13:18Balls into bins solution
- 13:58Code
- 14:37Results
- 15:57Model latency
- 18:10Model latencies
- 20:31greedy algorithm
- 20:44join algorithm
- 21:38randomize algorithm
- 22:00jsq
- 22:40Conclusion
- 24:26Questions
related talks
35mSystems Scale Summer Q&A Week 1
40mEverything Will Flow
21mSvalinn: Overload Control in Large-Scale Servers with Multiple Resource Bottlenecks
25mAvoiding Overload in Distributed Systems
16mTowards Optimal Rack-scale μs-level CPU Scheduling through In-Network Workload...
25mApproximating Fair Queueing on Reconfigurable Switches
02h 42m!!! Do not publish or share !!!!
19mLook Before, Before You Leap: Online Vector Load Balancing with Few Reassignments