Beauty and the Beast: your application and distributed systems
summary
Shows concrete patterns for safe retries: use idempotent operations (MongoDB unique-pending-set or Elasticsearch versioned updates) and prefer client-scoped guarantees like client-centric consistency (MongoDB causal sessions). Demonstrates a two-node cluster with wait_for_active_shards=2, explains election behavior, heartbeat and server_selection_timeout tuning, and recommends chaos tests—kill a node, change replica roles, enable auth, vary latency—plus bounded retry policies (e.g. retry count or time limit) instead of blind retries.
AI-written summary · about
chapters (13)
- 0:00Start
- 8:10Define a distributed system Consensus Synchronization Replication Consistency Fault tolerance
- 9:01What is a Distributed system?
- 10:331. Communication between components is invisible.
- 11:33False assumptions The network is reliable The network is secure The network is homogenous The topology does not change Latency is zero Bandwidth is infinite Transport cost is zero. There is one administrator.
- 14:24Whether to commit a transaction to a database. Choosing a new leader.
- 20:55Deviation of Staleness Ordering of update operations
- 23:47Eventual Consistency
- 25:46Types of faults Transient Intermittent Permanent
- 31:19The client doesn't know what your application is doing. It's not always able to determine what is safe to retry.
- 34:51Idempotent operations
- 41:53Testing against a single server. running on localhost isn't sufficient.
- 46:04Questions
related talks
31m"Beauty and the Beast: Applications and Distributed Systems"
34mDistributed Systems - What Can Go Wrong and How to Get It Right
44mDistributed systems and scalability rules
35mDesign patterns for application resiliency
01h 38mBuilding Reliable Distributed Systems
53mResiliency and Availability Design Patterns for the Cloud
25mGrowing Distributed Systems
36mDesigning for failure - architecting fault-tolerant system