Scaling APIs to Billions: Extreme-Scale Reliability
summary
Claims concrete wins and concrete tools: CDN edge caches serve stale responses with background refresh to guarantee sub-50ms edge responses and reduce origin load by 95% origin traffic reduction; request coalescing uses single-flight request coalescing to collapse 10,000 concurrent misses into one backend call; multi-region active-active with regional autonomy and CRDTs for conflict-free replication avoids cross-region stalls; consistent hashing with virtual nodes, Envoy-style service mesh, tail-based tracing, and predictive autoscaling are the actual mechanisms and tradeoffs shown.
AI-written summary · about
chapters (10)
- 0:00Introduction and Speaker Background
- 0:30Transition to Extreme Scale Reliability
- 2:08Case Studies: Netflix, Uber, and RevenueCat
- 3:30Four-Layer Scaling Framework
- 4:53Edge Layer Strategies
- 6:23Request Coalescing and Traffic Patterns
- 8:43Rate Limiting and Multi-Region Architecture
- 11:16Sharding Techniques and Service Mesh
- 13:12Observability and Predictive Autoscaling
- 15:14Future Challenges and Conclusion
related talks
20mSoftware Performance Optimization and Scaling
44mAvailability, Latency and Cost: Withstanding Outages
52mEdge Side APIs : Fast and Reliable Hypermedia APIs
40mThe Art of Embracing Failure at Scale
15mBuilding Self-Healing Cache Infrastructure
23mHow to edge cache GraphQL APIs
02mVelocity 2014
32mBeyond Optimization: Resilient Cloud Microservices at Scale