Accelerating Performance: Mastering RocksDB for High Speed Data Processing
summary
Arcesium uses RocksDB via a JNI bridge to process millions of events with a claimed sub-millisecond latency. Implementation details: LSM-backed SST files, memtables, block/OS caches, bloom filters, and column families for secondary indexes; range delete plus scheduled leveled compaction removes tombstone bloat. Real problems solved: custom key encoding for numeric sort order, off-heap growth cured by switching the memory allocator, and backups/monitoring built around RocksDB’s backup APIs and compaction windows.
AI-written summary · about
chapters (5)
- 0:00Introduction to RocksDB
- 6:22Why RocksDB was chosen
- 15:13Challenges in production
- 25:49Tips for production use
- 28:59Q&A and summary
related talks
52mRocksDB Performance and Reliability Practics
20mEnabling Machine Learning through Real-Time Data Processing
57mRocksDB
37mMoving on from RocksDB to something FASTER
12mSpanDB: A Fast, Cost-Effective LSM-tree Based KV Store on Hybrid Storage
51mSix New Important RocksDB Features And Planned Works
24mPerformance tuning in BlueStore & RocksDB
15mADOC: Automatically Harmonizing Dataflow Between Components in Log-Structured Key-Value