RubbleDB: CPU-Efficient Replication with NVMe-oF
summary
RubbleDB uses NVMe-oF target offload to ship compaction outputs once from a primary to secondaries, avoiding redundant compactions and offloading disk/network datapaths to the NIC. It implements a pre-allocated file pool plus a bitmap so shipped SST files are visible without filesystem races, and enforces a memtable-ID partial ordering to preserve consistency across concurrent writes. Built on RocksDB, it reports up to 1.9× throughput, 66% less compaction CPU, and up to 93.4% lower tail latency at the cost of ~11 GB extra network traffic.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 0:33Background compaction jobs
- 2:05CPU consumption
- 3:03Are redundant compactions necessary
- 3:20Remove redundant compactions
- 4:44Challenges
- 7:02New Challenge
- 8:55SSD File Preallocation
- 14:13Partial Order
- 16:33Evaluation Results
- 16:48Evaluation Methodology
- 17:09Endtoend throughput
- 18:22Latency
- 20:11Tradeoff
- 21:37Conclusion
related talks
24mSplinterDB: Closing the Bandwidth Gap for NVMe Key-Value Stores
57mRocksDB
54mEVCache: Lowering Costs for a Low-Latency Cache with RocksDB
32mLow Latency Serving of Offline Data: Efficient, Safe, and Reliable Data...
26mLeveraging Moore's Law to Optimize Database Performance
19mSILK: Preventing Latency Spikes in Log-Structured Merge Key-Value Stores
21mMatrixKV: Reducing Write Stalls and Write Amplification in LSM-tree Based KV Stores
24mPerformance tuning in BlueStore & RocksDB