Achieving Rapid Response Times in Large Online Services
summary
Describes concrete techniques that pull down 99th/99.9th-percentile latency in shared clusters: fine-grained partitioning (typically 10–100 pieces per machine), selective replication and removing slow nodes via latency-induced probation, plus quick adaptations. Implements backup requests and cross-server cancellation in Bigtable, GFS/distributed file system and query serving; sending backups at 2–10 ms cuts long tails dramatically, even under heavy MapReduce load, with only about a 1% extra I/O cost, and yields much faster tail latency than no backups.
AI-written summary · about
chapters (16)
- 0:00Intro
- 0:37Large Fanout Services
- 4:06Basic Latency Reduction Techniques
- 6:06Tolerating Faults vs. Tolerating Variability
- 7:36Latency Tolerating Techniques
- 8:41Fine-Grained Dynamic Partitioning
- 9:56Load Balancing
- 10:27Speeds Failure Recovery
- 11:07Selective Replication
- 12:09Latency-Induced Probation
- 14:13Data Independent Failures
- 16:12Canary Requests
- 18:52Backup Requests Effects
- 22:03Backup Requests: Bad Case
- 25:48Backup Request Variants
- 26:30Conclusions
related talks
10mConsiderations for a large-scale low-latency system
32mLow Latency Serving of Offline Data: Efficient, Safe, and Reliable Data...
31mPatterns of Low Latency
25mAvoiding Overload in Distributed Systems
38mHazelcast That Could: Getting Lightning-Fast Responses at Hundreds of RPS
16m"Performance Testing: ..."
36mMake It Fast
19mKeeping High-Traffic Systems Running When Things Go Wrong