Towards consensus on distributed consensus #bbuzz
summary
Explains when distributed consensus is actually required versus cheaper coordination, using ZooKeeper, etcd/Consul and Raft examples and two production patterns: replicated state machines via atomic broadcast ↔ consensus and configuration masters (Kafka, BookKeeper) that store ISRs in ZooKeeper with versioned writes as a compare-and-swap is consensus-level primitive. Presents FLP impossibility and partial synchrony fix, reiterates need for 2f+1 replicas for f crash tolerance, and shows read/write registers (KV stores) do not require consensus.
AI-written summary · about
chapters (8)
- 0:00Introduction and motivation
- 2:55What is distributed consensus
- 11:35Impossibility results and practical solutions
- 16:00State machine replication and atomic broadcast
- 22:03Kafka and BookKeeper examples
- 30:13Consensus in coordination systems
- 34:56Summary and takeaways
- 36:00Q&A
related talks
42mConsensus Algorithms in Distributed Systems (Theory and Practice)
48mDistributed Consensus Algorithms
40mConsensus Algorithms in Distributed Systems
51mDo Not Reimplement Distributed Consensus Algorithms
52mThe Computer Science behind a modern distributed data store
37mUnderstanding Distributed Consensus in etcd and Kubernetes - Laura Frank, CloudBees
44mCluster Consensus: When Aeron Met Raft
43mAll Together Now: An Introduction to Distributed Consensus