Lessons Learned From Architecting the Highest-scale Operational Systems in the World
Talkat KubeCon + CloudNativeCon Europe 2025 · on Wednesday, 2025-04-02· at 11:15· Level 0 | ICC Capital Hall | Room J·
summary
Fastly runs the global edge that serves about 1M requests per second for open‑source packages and peaks near 40M req/s with ~400+ Tbps capacity, and the platform lesson is relentless focus on outliers — P99/P999 tail latency matters. Operational rules: treat code and config the same, avoid global instantaneous state changes, require canary crash investigation, and enable automated rapid rollback. Example fix: removing unexpected file IO from an event loop cuts tail latency significantly and rolls out globally after controlled verification.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 3:57Festol tools for platforms
- 14:09Platform consistency promise
- 15:11High percentiles focus
- 18:21Outcomes over innovation
- 20:51Preventing global outages
- 24:01LLM incident analysis
- 25:50Correct abstractions
related talks
01h 01mLittle things and big things: lessons for building large systems.
56mMastering Cloud Scalability and System Designs Deep dive
34mDistributed Systems - What Can Go Wrong and How to Get It Right
19mEngineering for the Long Game
20mBetter Resilience Through Self Tuning
48mMoving Fast at Scale
01h 02mTools for simplified large scale systems
46mGo Faster -Impact on Technology, Process and Organization