Sliding Look-Back Window Assisted Data Chunk Rewriting for Improving Deduplication...
summary
Proposes a rewrite system combining FC-RC (flexible container reference-count thresholds) with a sliding look-back window plus candidate cache and recipe cache to decide per-chunk rewrites, implemented in simulation on six FSLA/Stony Brook traces. Claims this reduces container reads and yields much faster restores: a reported speed factor ~3.7 versus ~1.2 for container capping, about 30% higher restore throughput than capping, while keeping deduplication loss small and saving roughly 75% of space versus no-rewrite baselines.
AI-written summary · about
chapters (18)
- 0:00Intro
- 0:58Deduplication Process
- 4:08Data Chunk Rewrite Schemes
- 4:52Challenges of Rewrite
- 5:43Introduction of Container Capping
- 8:08Fixed Capping Level
- 9:17Fixed Segment Cutting Issue
- 10:20Flexible Container Referenced Count based Design (FCRC)
- 11:44Set Two Bounds
- 13:44Sliding Look-back Window Assisted Rewrite
- 14:58Architecture
- 16:07Example
- 19:52Rewrite Policy
- 20:12Experiment Setup
- 21:59Capping vs. LBW
- 23:42Speed Factor Comparison
- 24:05Detailed Comparison
- 24:26Conclusions and Future Work
related talks
22mLoopDelta: Embedding Locality-aware Opportunistic Delta Compression in Inline...
23mDependence-Preserving Data Compaction for Scalable Forensic Analysis
09mLogCrisp: Fast Aggregated Analysis on Large-scale Compressed Logs by Enabling...
34mA journey of scaling high-volume low-latency stateful processing
32mCan local-first really scale at the edge?
28mA look at Flink’s internal data structures & algorithms for efficient checkpointing
21mRun Interactive Queries with Apache Flink
41mMaximising Data Storage Efficiency with Ceph as a Caching Layer for Tape Archiving