SmartDedup: Optimizing Deduplication for Resource-constrained Devices
Talkat USENIX ATC '19 · on Thursday, 2019-07-11· at 15:50· USENIX ATC Track II: Grand Ballroom VII–IX·
summary
SmartDedup implements a hybrid deduplication system on EXT4 and F2FS, prototyped on Nexus 5X and Raspberry Pi 3, that combines in-memory + on-disk fingerprint stores and symbiotic in-line and out-of-line deduplication with adaptive throttling. It uses a 12-bit prefix tree, migrates fingerprint groups to disk to keep memory to ~3.5 MB, and adds a page-cache index to avoid duplicate reads. Evaluations show ~31% write and 32% read throughput gains (FIO, 25% dedup), 70.9% fewer flash writes (72.4% dedup trace), 45% space saved (46.1% dedup), and adaptive mode cuts power 14% for 8% dedup loss.
AI-written summary · about
chapters (17)
- 0:00Intro
- 0:29Resource Management on Edge and lot
- 0:57Deduplication to the Rescue?
- 1:40Smartphone Trace Analysis
- 2:30More In-depth Analysis
- 3:37Challenges to Device Deduplication
- 4:08SmartDedup-A Smart Deduplication Solution for Smart Devices
- 5:03In-memory Fingerprint Store
- 5:46On-disk Fingerprint Store
- 6:17Fingerprints Migration
- 7:04Hybrid Deduplication
- 9:08Optimized Read Path
- 10:56Evaluation
- 12:31FIO (Resource Usage)
- 14:19Adaptive Deduplication (on Nexus)
- 15:11Conclusions
- 17:53Trace Replay (on Nexus)
related talks
33mSSDFS: Flash-friendly File System with Highly Minimized GC Activity, Diff-on-write, & Deduplication
38mEROFS, an Introduction and our Smartphone Practice
46mMaximizing SD Card Life, Performance, and Monitoring with KrillKounter
14mModernizing File System through In-Storage Indexing
13mInsight of Linux Distro Optimization for RISC-V: Using VisionFive 2 Debian as Exam...
25mReducing Android Boot Times: Evaluating Proactive File Prefetching
01h 01mBoosting Software Efficiency : A Case Study of 100% Performance Improvement in ...
54mBetter Alignment of Flash Storage to Mobile System Behavior