Quiver: An Informed Storage Cache for Deep Learning
summary
Quiver is a cache layer co‑designed with PyTorch that uses secure hash-based addressing to let different jobs and accounts reuse identical files, implements substitutable cache hits (look up many indices and return any cached subset) plus coordinated eviction and double buffering, and runs a cache server, client and manager. Prototype evaluation on ResNet‑50/ImageNet, Inception v3/OpenImages and DeepSpeech2/LibriSpeech shows cluster GPU throughput improvements up to 2.3x, with wins even when only ~20% of a dataset fits in cache.
AI-written summary · about
chapters (21)
- 0:00Intro
- 0:20Deep Learning: Important systems workload
- 1:39Example workload
- 3:39Quiver: key ideas
- 4:14Structure
- 4:31Background: Deep Learning
- 5:22A cache for DLT jobs
- 7:22Thrashing proof partial caching
- 9:13Job heterogeneity and caching
- 10:28Secure Inter-Job Data access
- 11:32Architecture of Quiver
- 13:06Cache Access
- 13:47Hash digest and Partition
- 14:39Co-operative miss handling
- 17:42Implementation
- 18:07Evaluation Setup
- 18:36Impact on accuracy
- 19:52Throughput increase because of quvier
- 21:17Co-ordinated eviction in action
- 22:12Benefit aware caching
- 24:16Summary
related talks
17mDroidSpeak: KV Cache Sharing Across Fine-tuned Model Variants
15mKV Cache: Accelerating AI Inference on Intel CPU
18mSHADE: Enable Fundamental Cacheability for Distributed Deep Learning Training
25mTo Cache or Not To Cache? a Tiered KVCache Storage System for Agent ...
14mHyCache: Hybrid Caching for Accelerating DNN Input Preprocessing Pipelines
26mUnlocking Scalable Distributed Training With Arrow Data Cache on Kubernetes
15mECHO: Efficient KV Cache Offloading with Lossless Prefetching for Serving Native...
15mBidaw: Enhancing Key-Value Caching for Interactive LLM Serving via Bidirectional...