Performance tuning in BlueStore & RocksDB
summary
Examines concrete BlueStore/RocksDB choices: memtable sizing, flush behavior and how 256MB memtable default and splitting into multiple memtable slices changes write amplification and latency; SSD allocation granularity (16KB default) and its effect on small randomized writes; compaction level movement that makes level0 to level1 cause heavy amplification; results from 4K vs 16K random-write tests showing higher latency at smaller writes and a best-practice suggestion of larger memtable slices (~512MB total per OSD) to reduce amplification and improve throughput.
AI-written summary · about
chapters (12)
- 0:00Introduction to BlueStore
- 1:11BlueStore architecture overview
- 3:05Write path in BlueStore
- 6:25Small writes handling
- 8:19Data flow in BlueStore and RocksDB
- 10:30Latency testing with fio
- 12:16Random write performance issues
- 13:52Tuning memory table sizes
- 16:51Impact of memory allocation
- 18:26Random write amplification analysis
- 20:45Write amplification in RocksDB
- 22:17Future work and Q&A
related talks
52mRocksDB Performance and Reliability Practics
46mBlueStore V2 - an Evolutionary Step Forward
57mRocksDB
15mImproving Performance of Flash Based Key-Value Stores Using Storage Class Memory...
13mClosing the B+-tree vs. LSM-tree Write Amplification Gap on Modern Storage Hardware...
51mSix New Important RocksDB Features And Planned Works
29mAccelerating Performance: Mastering RocksDB for High Speed Data Processing
34mWhat We Talk About When We Talk About Disk IO