Designing for failure
summary
Claims that resilient systems use concrete tactics not platitudes and demonstrates them with Go libraries and patterns that actually run. Uses circuit breaker with three retries and exponential backoff to stop cascading failures, separates client and server timeouts, enforces bulkheads so one failing downstream does not kill the whole service, and implements an outbox pattern instead of making the system depend on Kafka availability. Cites monitoring with Prometheus and OpenTelemetry and notes 39 million flights 2019 with 8 catastrophes to argue failure needs design.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:04Why system resilience matters
- 2:55Handling timeouts
- 7:23Using circuit breakers
- 11:00Load distribution
- 12:30Resource isolation
- 14:40Service health checks
- 16:30Message broker patterns
- 19:21Traffic control and rate limiting
- 21:08Scaling challenges
- 22:34Service mesh examples
- 24:48Metrics and monitoring
- 26:33Distributed tracing
- 28:18Q&A
related talks
35mDesigning for failure
42mDesigning for failure
55mGoing Resilient
36mDesigning for failure - architecting fault-tolerant system
37mResilience Design Patterns
53mResiliency and Availability Design Patterns for the Cloud
57mResilience by Design
29mThe art of being resilient. How to handle failures gracefully in Distributed Systems.