Patterns of Low Latency
summary
Low latency demands concrete choices: measure percentiles with HDR histograms or ECDFs and avoid coordinated omission; treat latency as distribution not average. Reduce tail by three strategies—avoid data movement (collocate, replicate, cache; New York–London RT ≈57 ms, memory ~100 ns), avoid work (O(1) structures, avoid dynamic allocs and demand paging), and avoid waiting (disable Nagle, CPU isolation, disable swap, busy polling, use iouring/XDP/DPDK). Hide latency via parallelism and hedging and use single-producer single-consumer queues for wait‑free handoff.
AI-written summary · about
chapters (9)
- 0:00Introduction and agenda
- 1:28Why latency matters
- 3:57Measuring and visualizing
- 8:30Reducing: Data movement
- 12:27Reducing: Avoiding work
- 18:13Reducing: Avoiding waiting
- 24:36Hiding latency
- 26:51Tuning the system
- 30:10Summary and conclusion
related talks
50mDesigning for Performance
10mConsiderations for a large-scale low-latency system
49mResponding in a Timely Manner
52mMeasuring and Optimizing Tail Latency
32mSoftware Performance: A Shape Not a Number
49mHigh Throughput and/or Low Latency
31mLow-Latency Solutions for Resource-Constrained Environments
28mAchieving Rapid Response Times in Large Online Services