Incident Command at the Edge
summary
Fastly runs a documented Incident Command system: a shared infinite commander rotation coordinates impact assessment, calls subject-matter on-call experts, and can lock deployments; incidents use a simple severity 0–3 scale and are timeboxed with a 10-minute timebox for escalation. Teams post timelines to Jira within 24 hours, run blameless postmortems, hold tabletop exercises every six months, let engineering groups choose monitoring tools, and publish transparent status updates (Fastly updated customers before Amazon during the S3 outage). Rotation length, on-call training and responder checklists are concrete levers for reducing repeat incidents.
AI-written summary · about
chapters (12)
- 0:00Start
- 1:1436 POPs; 5 Continents; -10T/sec Network
- 4:13Incident Types
- 6:11Developing the Framework
- 8:28Severity Levels
- 10:36Incidents are made of People
- 14:23Incident Commander
- 18:16Communication: Wide Collaboration
- 22:53Continuous improvement
- 24:22Incident Response Process
- 25:02Exercises
- 25:48Lessons
related talks
20mHow to Have an Operational Incident
28mHow to Have an Operational Incident (A Crash Course)
27mManaging Incidents
30mHow To Have An Operational Incident (a crash course)
33mDon't Panic! Effective Incident Response
32mSustainable Incident Management
27mLessons from Incident Management and Postmortems at Atlassian
35mIf I Can Do It on an Ambulance, You Can Do It in an Office: Scalable Incident