Distributed Computing Dealing with Time and Failure in the Wild
summary
Compares concrete implementations: building a pipeline to enrich/transform/index a billion journals using Storm’s spouts and bolts (topology, shuffle/field groupings, at‑least‑once retries, idempotence required) versus an Akka design with actors and supervision (mailboxes, routers, dynamic hierarchies, restart strategies). Explains how failures are handled (replay versus supervisor restart), when throughput falls for stronger once‑only semantics, and why monitoring (mailbox sizes) and eventual‑convergence types (CRDTs / “eventual ACID”) matter.
AI-written summary · about
chapters (10)
- 0:00Distributed real-time processing intro
- 5:06Storm basics and topology design
- 16:32Fault tolerance in Storm
- 25:02Akka and actor model
- 35:24Storm vs Akka differences
- 42:13Reactive manifesto and Akka streams
- 43:17Time and state in systems
- 46:25Functional programming role
- 47:05Eventual consistency and CRDTs
- 48:52Q&A
related talks
40mBrace yourself! Storm is coming!
27mDistributed Systems, Like It or Not
29mHorrors of Distributed Systems
45mDo Distributed Systems they said. It'll be fun they said.
44mThe Art and Benefits of Computing Eventfully
25mGrowing Distributed Systems
02h 18mDistributed Systems: In the Daily Life of the Data Engineer
56mFrom Unreliable to Resilient: Mastering Distributed Systems