Differentiated Key-Value Storage Management for Balanced I/O Performance
summary
DiffKV builds a prototype on TiKV/Titan that splits keys and values and uses a V3 multi-level value tree plus fine-grained KV separation (small/medium/large) and new merges: compaction-triggered merge, laser merge, and scan-optimized merge. It reorganizes values during key compaction to avoid lookups, delays merges to cut frequency, and forces ordering when scans need it. Experiments show up to 3.8× inserts, 3.27× updates, 3.24× reads, comparable scans, and ~60% less time spent on value management.
AI-written summary · about
chapters (15)
- 0:00Intro
- 0:19Background
- 1:11LSM-tree: Basics
- 1:38Relaxing Fully-Sorted Ordering
- 2:19KV Separation Store keys and values separately .
- 2:57Trade-off Analysis Are the optimizations suitable for all conditions? • Relax fully-sorted ordering
- 3:40Our Contributions
- 4:41Differentiated KV Management
- 6:5401: Lazy Merge
- 8:2602: Scan-optimized Merge Problem: too many sorted groups within one level • Apply append-only merge policy
- 9:43Fine-grained KV Separation
- 10:43Experiments
- 11:11Microbenchmarks of DiffKV
- 11:48Impact of Merge Optimizations
- 12:24Conclusions
related talks
18mSplitKV: Splitting IO Paths for Different Sized Key-Value Items with Advanced
24mUsing a Distributed Key-Value Store
15mMitigating Resource Usage Dependency in Sorting-based KV Stores on Hybrid...
05mExploring Data Deduplication in LSM Tree-based Key-Value Stores (Work-in-Progress Report)
32mKeyvi the Key Value Index Optimized for Size and Speed #bbuzz
21mMatrixKV: Reducing Write Stalls and Write Amplification in LSM-tree Based KV Stores
11mAdapting TiKV for Cloud Storage
18mPreview: KV-Stores and Databases