3L-Cache: Low Overhead and Precise Learning-based Eviction Policy for Caches
summary
3L-Cache is an object-level learning eviction policy that uses GBM prediction, sampled training windows, bidirectional sampling, and heap+hash matching to cut CPU work while improving misses. It reports 60.9% lower CPU vs HALP and 94.9% vs LRB, raises eviction ratio to ~50% by prioritizing nonpopular objects, and limits training samples to 32k–122k to stay lightweight. Evaluation on 4855 traces yields lowest byte and object miss ratios among learned policies while incurring only 2.6×–6.4× the CPU of LRU depending on cache size.
AI-written summary · about
chapters (7)
- 0:00Cache eviction policies
- 1:45Learning-based eviction importance
- 3:53Object-level learning challenges
- 6:23Optimizing learning-based eviction
- 7:063L-Cache design
- 11:503L-Cache performance and overhead
- 14:393L-Cache summary
related talks
16mGL-Cache: Group-level learning for efficient and high-performance caching
11mLearning Cache Replacement with CACHEUS
20mLearning Relaxed Belady for Content Distribution Network Caching
34mVLLM KV Cache Management: From Cache Reuse To Agent Scenario Optimization
16mReinforcement Learning-Based SLC Cache Technique for Enhancing SSD Write
16mHALP: Heuristic Aided Learned Preference Eviction Policy for YouTube Content Delivery...
28mPagedAttention: Revolutionizing LLM Inference with Efficient Memory Management
20mAccelerating Model Loading in LLM Inference by Programmable Page Cache