Data-driven Postmortems
summary
Claims that monitoring must meet four concrete criteria — well understood, sufficiently granular, tagged/filterable, long‑lived — and that failure to do so hides real incidents: cloud metrics at one‑minute rollups turn a 46% CPU spike into 12%. Notes Datadog ingestion scale as trillions of points of data every day. Recommends concrete artifacts: dashboards/screenshots, Slack archives, a five‑part postmortem template, company‑wide emails, monthly reviews and actionable tickets tied to fixes.
AI-written summary · about
chapters (29)
- 0:00Intro
- 0:42Who am I
- 1:58Doing change
- 2:26Good metrics
- 5:49Tags
- 6:56Longlived metrics
- 7:12Machine learning
- 7:43Technical issues
- 7:56Dont postmortem
- 8:26Who collect data
- 9:58Why collect data
- 10:56Write it down
- 11:51Include pictures
- 12:22When to collect data
- 13:27Data Skew
- 15:09Datadog Postmortems
- 15:55Postmortem Template
- 16:22Example
- 16:45Impact on customers
- 16:56How was this detected
- 17:33How long did it take
- 17:45Take screenshots
- 17:55How did we respond
- 18:35Slack
- 19:21Learnings
- 19:34Why did it happen
- 19:46How to prevent it
- 20:33Working with other teams
- 20:58Resources
related talks
51mEmbracing Failure and Learning from Our Mistakes with Effective Post Mortems
22mAfter Death
10mSimplify your postmortems and focus on scaling
31mPost-mortems: Building better software together.
42mArchitecting a Technical Post Mortem
33m100% Visibility
12mUnlocking the Secrets of Client Data
12mUsing postmortems to break out of infinite loops