The hows and whys of a distributed SQL database
summary
Describes a working open-source distributed SQL engine (CockroachDB) that combines ordered range sharding (ranges ≈64–128MB) with per-range replication (typically three replicas) using Raft consensus to commit via a single group write, and MVCC plus conflict detection for serializable transactions. Gives concrete trade-offs: hundreds of thousands of ranges for terabytes of data, one network hop to quorum commits, complexity in rebalancing and member changes, and practical engineering techniques that make the system durable and strongly consistent.
AI-written summary · about
chapters (38)
- 0:00Introduction
- 1:28History
- 1:49Primitive databases
- 2:52The relational model
- 4:09Objectoriented databases
- 5:20Database sharding
- 6:54No sequel databases
- 8:06Distributed sequel databases
- 9:26Why these systems are being built
- 10:05The two primary systems
- 10:25The question of how
- 11:00Disclaimer
- 11:36Data Distribution
- 14:16Order Preserving
- 15:19Range Indexing
- 16:11Range Scan
- 16:22When to Split
- 18:43When to Up Replicate
- 19:42Primary Secondary Replication
- 21:49Eventually Consistent
- 23:18Distributed Consensus
- 25:04Raft
- 27:03When things dont go well
- 28:12Engineering complications
- 29:20Transactions
- 29:43Consistency
- 30:56Atomicity
- 33:00MVCC
- 34:35No sequel transactions
- 35:21Distributed sequel
- 37:06Starting a transaction
- 38:34Conflicts
- 38:47Write Conflict
- 38:59Detect Conflict
- 40:01Push or abort
- 40:11Intent
- 40:26Summary
- 41:45Outro
related talks
48mBuilding mission-critical applications with CockroachDB
52mCockroachDB
24mNavigating Distributed Databases: Challenges & Opportunities
01h 11mFearless global transactions with
37mBuilding a Cloud-Native SQL Database
26mNewSQL: concepts, realities and when you should embrace it
22mBuilding CockroachDB for performance at scale
37mRunning a Cloud-Native SQL Database on Mesos