Latency-Aware Caching with Delayed Hits: From Bursty Traffic to Pipeline Architectures

on YouTube ↗16m

" summary

Proposes an adaptive pipeline cache that composes simple eviction blocks (LRU, LFU, etc.) into ordered stages with configurable quanta and a runtime hill‑climbing allocator, plus a new burstiness block LBU and the combined RFB policy. Evaluated on 13 real traces (IBM, Twitter, Meta) with measured remote latencies; RFB cuts average request latency by ~10% versus the best prior algorithm, yields the lowest stddev across bursty and non‑bursty workloads, and outperforms the best static configuration when workloads change.

AI-written summary · about

" chapters (14)

1. 0:00Introduction
2. 1:03Latency-aware caching overview
3. 2:27Pipeline cache architecture
4. 3:55Processing and eviction in pipeline cache
5. 4:38Configuring pipeline blocks
6. 5:21Delayed hits problem
7. 7:05Introducing delayed hits
8. 8:30LBU policy for bursty workloads
9. 9:55Experimental setup
10. 11:26Performance comparison
11. 12:30Adaptive behavior under workload changes
12. 13:37Adaptive allocation analysis
13. 15:25Summary and future work
14. 15:45Q&A