Light-Dedup: A Light-weight Inline Deduplication Framework for Non-Volatile...
summary
Light-Dedup is a lightweight inline deduplication framework for NVM file systems that combines a non-cryptographic hash with speculative prefetch–driven byte-by-byte verification (LRBI) and an in-NVM region-organized metadata table (LMT). A Linux/Nova prototype on 256 GB DCPM demonstrates 1.01–8.98× I/O throughput versus prior NVM dedupe designs, shows speculative prefetch gains of 0.3–118%, and drops metadata read/write amplification from 19.35×/9.86× to 6.10×/3.43×, indicating the techniques work at scale.
AI-written summary · about
chapters (11)
- 0:00Light-Dedup and NVM challenges
- 2:50NVM deduplication limitations
- 3:53Deduplication read latency issues
- 5:26In-block prefetch for latency hiding
- 7:14Cross-block prefetch for parallelism
- 9:44Redundant block identifier design
- 10:49Metadata table design challenges
- 11:33Light metadata table for NVM
- 13:24Light-Dedup architecture
- 14:29Evaluation results and gains
- 15:31Summary and future
related talks
12mThe Dilemma between Deduplication and Locality: Can Both be Achieved?
05mExploring Data Deduplication in LSM Tree-based Key-Value Stores (Work-in-Progress Report)
18mDon't Maintain Twice, It's Alright: Merged Metadata Management in Deduplication File...
26mReaping the performance of fast NVM storage with uDepot
43mThe Forefront of the Development for NVDIMM on Linux Kernel
33mSSDFS: Flash-friendly File System with Highly Minimized GC Activity, Diff-on-write, & Deduplication
17mShieldReduce: Fine-Grained Shielded Data Reduction
17mPreview: Persistent Memory