ADOC: Automatically Harmonizing Dataflow Between Components in Log-Structured Key-Value
summary
Claims that LSM write stalls come from a universal data overflow — imbalanced input vs output across memtable, L0 and deeper compactions — and not any single resource. Implements ADOC that auto-tunes threads and batch size using an AIMD-like controller on RocksDB, measures across devices and workloads, and reports up to 87.9% shorter stall duration, up to 322.8% higher throughput vs autotuned RocksDB, and up to 66% higher throughput vs SILK while SILK uses ~20% more DRAM.
AI-written summary · about
chapters (5)
- 0:00Introduction to LSM systems
- 2:02Analyzing write stalls
- 5:46The data overflow concept
- 10:32Adoc tuning framework
- 14:19Conclusion and summary
related talks
19mSILK: Preventing Latency Spikes in Log-Structured Merge Key-Value Stores
29mAccelerating Performance: Mastering RocksDB for High Speed Data Processing
24mPerformance tuning in BlueStore & RocksDB
21mMatrixKV: Reducing Write Stalls and Write Amplification in LSM-tree Based KV Stores
15mRemoving Double-Logging with Passive Data Persistence in LSM-tree based Relational...
34mWhat We Talk About When We Talk About Disk IO
18mPreview: KV-Stores and Databases
25mAC-Key: Adaptive Caching for LSM-based Key-Value Stores