Fault isolation using shuffle sharding
summary
Explains how splitting customers across combinations of workers limits blast radius and gives hard numbers: with eight workers and two-per-customer there are 28 unique combos so the worst-case scope can be 1/28 impact with eight nodes. Shows a working option: Route 53 infirma library (Java, Maven) that pre-bakes decision trees to eject failed endpoints fast. Compares ordinary sharding (25% in a four-shard example) to shuffle sharding and describes tradeoffs: extra spare capacity and reduced cross-customer failure propagation.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:17Reliability definition
- 3:24Sharding workloads
- 4:50Basic sharding example
- 7:21Shuffle sharding benefits
- 10:31Route 53 implementation
- 12:16Scaling shuffle sharding
- 13:46Hands-on resources
- 14:51Conclusion
related talks
28mSharding a GraphQL Gateway for Blast Radius Reduction
46mDatabase Sharding the Right Way Easy, Reliable, and Open source
15mDistributed Caching & Queueing in the Cloud
33mShopify in Multiple Datacenters
34mSplit That Monolith: a Java on Cloud Foundry Story
31mThe Anatomy of a Cascading Failure
04mThayne McCombs Interview
44mScaling Horizons Effective Strategies for Wix's Scaling challenges