XLL: Cross-Layer Logging for Data Deduplication in Consensus-Based Storage
summary
A shared append-only log called XLL replaces separate consensus and KV write-ahead logs and stores values once while the KV LSM keeps only locators; integrated into TiKV XL this yields a 5.5x increase in write throughput on write-heavy YCSB workloads and a 73% reduction in write amplification versus baseline TiKV, preserves read performance, and recovers state via a two-phase log replay that rebuilds lost memtable updates.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:23Multi-layer write amplification
- 4:51Key-value separation systems
- 6:16Design challenges in XLL
- 8:43Efficient read algorithm
- 9:04Crash recovery in XLL
- 10:49Evaluation setup
- 11:32End-to-end latency results
- 12:35Write amplification comparison
- 13:41YCSB performance analysis
- 14:21Conclusion
related talks
14mLoLKV: The Logless, Linearizable, RDMA-based Key-Value Storage System
05mExploring Data Deduplication in LSM Tree-based Key-Value Stores (Work-in-Progress Report)
21mMatrixKV: Reducing Write Stalls and Write Amplification in LSM-tree Based KV Stores
11mAdapting TiKV for Cloud Storage
18mPreview: KV-Stores and Databases
33mScaling KV Caches for LLMs: How LMCache + NIXL Handle Network and Storage...
13mDifferentiated Key-Value Storage Management for Balanced I/O Performance
45mOK Log: Distributed and Coördination-Free Logging