Fighting AI with AI
summary
Incident.io uses concrete tooling: a small eval CLI lets coding agents read and edit huge eval YAMLs to create a red‑green runbook that proves fixes and checks regressions; every UI debugging view is serialized as a filesystem package drop‑in for Claude Code so an agent traces the prompt hierarchy to the exact prompt to change; fleet analysis spins up 25 agents in parallel, then clusters their findings to surface systemic failure patterns across customers.
AI-written summary · about
chapters (9)
- 0:00AI-driven incident management
- 1:31Debugging complex AI systems
- 3:38AI for eval datasets
- 7:04Agents with eval tooling
- 9:12Debugging multi-prompt AI systems
- 10:57UI tools to file systems
- 13:01Repeatable AI analysis pipelines
- 15:48Key takeaways
- 16:51Closing remarks
related talks
30mAI Agents for Incident Management
16mAI Agents for Incident Investigation: The Good, The Bad, and The Ugly
30mBeyond the Hype: Using AI Coding Agents Today
33mStrategic AI Agents for K8s SRE: Beyond Ad-hoc Prompting
19mYour Biggest User Isn't Human: Designing DevTools for AI Agents
30mArchitecting Integrity: Responsible Software Evolution in the AI Era
08mBuilding an AI Platform Your Engineers Actually Trust
35mThe Agentic AI Engineer