Measuring and Optimizing Tail Latency
summary
A tool measures thread activity at 1000 cycle granularity with 1% overhead and attributes 99th‑percentile latency to queuing, noise, or real work. Probabilistic reissue reduces noise-driven tails; judicious dynamic adaptive parallelization launches ~3–4 worker threads only for revealed long requests and shortens tails while often improving throughput. DVFS and emulated asymmetric multicore steer long requests to faster hardware, demonstrating roughly ≈10% energy savings for equal tail targets and extending sustainable requests‑per‑second before overload.
AI-written summary · about
chapters (24)
- 0:00Introduction
- 1:01We are impatient people
- 1:23Tail latency
- 1:37Big data centers
- 3:22Energy consumption
- 5:02Cost savings
- 6:06Efficiency
- 6:24Server Architecture
- 10:03Overview
- 11:36Long Requests
- 12:09Simplified Requests
- 12:31Prior State of the Art
- 14:45Memory Locations
- 15:15Shared Locations
- 16:57Log of Samples
- 20:50Configuration
- 22:22Program Counter
- 29:55Shortening Queue
- 30:12Parallelization
- 36:36Slow to Fast
- 39:00Pegasus
- 40:15Results
- 44:19Conclusion
- 47:27Questions
related talks
31mPatterns of Low Latency
42mWhen the OS gets in the way
49mResponding in a Timely Manner
19mCome Hell or Still Water: Alleviating Tail Latency in Cloud Block Store
30mQueueing Theory, In Practice: Performance Modelling in Cloud-Native Territory [I]
40mPerformance Testing Done Right
31mMeasuring the Impacts of the Preempt-RT Patch
49mHigh Throughput and/or Low Latency