ARKDB: The Key-Value Engine for Alibaba Cloud Storage Services
summary
ArkDB is a key-value engine built for Alibaba Cloud on top of Pangu that combines LSM ideas with Bw-tree–style page mapping to reduce compaction impact and scale partitions. Measured on ingestion after 100M updates, ArkDB shows about 2x throughput vs RocksDB and a write amplification 3.1 vs 9.5; the page-mapping memory stabilizes near ~10 MB with ~150–160k mapped pages out of ~500k total. Fast partition split doubles throughput for a 32 GB shard split into two 16 GB shards, with replay taking a few seconds.
AI-written summary · about
chapters (12)
- 0:00ARKDB and Alibaba Cloud intro
- 1:04ARKDB design goals
- 3:54Key-value store basics
- 6:05LSM-tree hardware issues
- 9:59ARKDB: LSM-tree meets B³
- 12:06ARKDB architecture
- 13:11Page mapping in ARKDB
- 15:39Checkpointing and compaction
- 18:27Garbage collection in ARKDB
- 19:31Fast partition splits
- 20:33Performance evaluation
- 22:43Challenges and future work
related talks
29mAccelerating Performance: Mastering RocksDB for High Speed Data Processing
27mMyRocks: best practice at Alibaba - MySQL Storage Engine
11mEvolution of Development Priorities in Key-value Stores Serving Large-scale Applications
12mSpanDB: A Fast, Cost-Effective LSM-tree Based KV Store on Hybrid Storage
25mAC-Key: Adaptive Caching for LSM-based Key-Value Stores
36mArcticDB, the OLAP antidote
57mRocksDB
30mWhat a serverless database means for users