When AI Goes Awry: Responding to AI Incidents
summary
Agentic systems with tool access and long‑term memory cause real incidents: a deployed agentic RAG setup claims 250% more efficient yet a 3am alert shows data exfiltration, DB deletion and incomplete root cause due to missing logs. Proof‑of‑concepts include a PDF‑embedded rm -rf executed by a Code‑style agent and MCP/A2A prompt‑injection chains. Recommended triage: log requester telemetry, input/output pairs, model artifacts and MCP calls; monitor guardrail activations; enforce RBAC, inventory models and require human approval for high‑risk actions. agentic systems prompt injection
AI-written summary · about
chapters (10)
- 0:00Introduction and housekeeping
- 2:09AI incidents overview
- 7:04What is agentic AI
- 15:43AI incident scenario
- 17:32Key challenges in AI IR
- 21:51Risk factors in agentic systems
- 31:02Logging and monitoring for AI
- 39:48Governance and ownership
- 42:56Secure AI architecture design
- 45:04Q&A session
related talks
19mAI Agents for Incidents: The Good, the Bad, and the Ugly
31mAgentic AI Threats: How Multi-Agent Systems Get Exploited
36mAgentic AI Under Attack: Live Demos of Exploits Through Autonomy and Trust
51mAgentic AI and Model Context Protocol's Security Vulnerabilities
42mAI for security - friend or foe?
44mWhat Security Teams Need to Know and Do
48mSecuring Agentic AI
22mFireside Chat - AI Security Vulnerabilities