Building Fault Tolerant Microservices
summary
Claims that microservices need aggressive timeouts and read/connect timeouts to avoid blocked threads, plus circuit breakers fail fast to prevent overload, and bulkhead semaphore or thread-pool handovers to cap concurrent calls. Concrete numbers: Avanza-scale example runs ~250 services and 1,000+ instances; five 9s per service across 1,000 instances collapses to ~87 hours/year downtime. Bulkhead sizing example: 40 rps × 0.1 s = 4 permits (add breathing room → ~7). Uses Hystrix for thread-pool handovers, CBs, metrics; monitoring focuses on timeout, rejected, short‑circuit and success rates.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 0:31The necessity of resilience
- 1:35Context and speaker background
- 2:53Monolith to microservices
- 5:17Cascading failures
- 6:53Designing for failure
- 8:17Thread exhaustion explained
- 10:33Pattern 1: Timeouts
- 12:46Pattern 2: Circuit breakers
- 15:28Default responses
- 19:24Pattern 3: Bulkheads
- 23:18Bulkhead sizing and logic
- 29:25Pattern 4: Thread pool handover
- 34:48Monitoring service calls
- 39:59Hystrix library implementation
- 43:08Conclusion
- 44:31Q&A
related talks
29mFault Tolerance is a Requirement Not a Feature - Adar Danait, Lilian Ernest, Citi
42mArchitecting for Failures in Microservices
43mFault tolerance – look, it’s possible!
23mMicroservices – Diptanu Choudhury
45mAvoiding Microservice Megadisasters
40mCAPtivating architecture: Navigating Distributed Systems and Microservices
44m3 common pitfalls in microservice integration and how to avoid them
46mSurviving microservices