Incidents Happen

on YouTube ↗34m

" summary

Argues that incident response borrows agile: timeboxed work, retrospectives and an incident commander role, and that aggressive automation beats manuals. Shows a demo wiring New Relic alerts to xMatters flow designer which creates Slack channel, Jira and ServiceNow tickets and posts a status page and conference bridge. Automated diagnostics surface a problematic RabbitMQ backlog — 7,500 messages in RabbitMQ — plus a recent Git/Jenkins deploy, and the system records timelines and SLA metrics for postmortems. The codeless flows run end to end and reduce manual toil, speeding detection, acknowledgement and resolution.

AI-written summary · about

" chapters (9)

1. 0:00Intro
2. 0:35Agenda
3. 1:01What are Incidents
4. 2:58Types of Incidents
5. 5:58Agile Principles
6. 8:52Scrum
7. 11:09DevOps
8. 13:37Reliability Engineering
9. 16:18Practice

" related talks

• 17mIncident Response @ Scale-Building a Next Generation SOCOmer Cohen · Black Hat Asia 2016
• 22mSmarter Disasters: End-to-End Automation for IncidentsKarthik Nilakant · SREcon18 Asia/Australia
• 28mFinal Bit: Automatic DevOps ManagementEvgeny Savitsky · DevOps Pro Moscow 2016
• 01h 04mIncident Management - Talk the Talk, Walk the WalkHila Fish · Copenhagen Developers Festival 2023
• 31mFuturistic Luxury Incident ResponseMandi Walls · SREday 2023 London
• 29mComposing the Response: Building an Incident Pipeline from ScratchGeet Pradhan · BSidesSF 2026
• 05mDevOps and incident management: A recipe for successDavid Hayes · Velocity 2017
• 45mFailure Happens: Improving Incident Response in Large-Scale OrganizationsDamon Edwards · LISA17