LogDevice: durable and highly available sequential distributed storage system
summary
Facebook’s LogDevice is a sequential, append‑only store that claims low tail latency (about 10 milliseconds) while handling huge throughput (measured in terabytes per second), built with separate sequencers and storage nodes, non‑deterministic copyset placement and epoched sequence numbers to avoid global contention. It uses a RocksDB-based LogDB with partitioned files that get dropped whole for retention, defaults to replication factor three, and performs fully distributed rebuilds; the system is implemented and running in production.
AI-written summary · about
chapters (26)
- 0:00Introduction
- 3:26Target use cases
- 4:16Use case: Graph index
- 5:54Use case: Scribex
- 8:54Performance requirements
- 10:46Consistency requirements
- 11:31Workload expectations
- 12:58Logs are simpler than files
- 14:27LogDevice API
- 15:45Design principles
- 17:23Writer sends an append request
- 18:03The sequencer
- 19:01Record numbering in a log
- 19:59The appender
- 20:49Concurrent appenders
- 21:18Non-deterministic copy placement
- 22:30Record format
- 28:08Data loss reporting
- 33:01LogsDB: Read amplification
- 33:37How to limit read amplification?
- 35:10Space reclamation in LogsDB
- 35:55Failure handling
- 36:39Rebuilding is many-to-many
- 37:29Many-to-many rebuilding
- 38:08Disaggregated clusters
- 39:28Server-side filtering
related talks
57mRocksDB
45mOK Log: Distributed and Coördination-Free Logging
12mSpanDB: A Fast, Cost-Effective LSM-tree Based KV Store on Hybrid Storage
01h 01mPanel: Facebook's Distributed Database Evolution
14mLoLKV: The Logless, Linearizable, RDMA-based Key-Value Storage System
30mData @Scale Seattle- Muthu Annamalai
52mRocksDB Performance and Reliability Practics
40mTaming Spiky Log Volumes: Maintaining Real-Time Log Accessibility with Kaldb