Attacking AI
summary
Seven-step methodology targets inputs, ecosystem, front-end model, prompt engineering, data stores, application layers and pivoting to impact; case studies show ASCII-encoded prompts bypass Amazon Rufus, uploaded files cause RAG data leak and blind XSS that harvests Microsoft cookies, and an agent prompt leak exposes Jira/Confluence API keys. Tests rely on toolchain like Burp Suite, XSS Hunter Express and Parseltongue, require 5–15 tries per case, and manual attackers bypass layered defenses far more reliably than current automation.
AI-written summary · about
chapters (20)
- 0:00Welcome to attacking AI
- 3:58Natural language prompt injection
- 7:05First try fallacy in AI security
- 8:29AI system architectures compared
- 11:43LLM assessment methodology
- 14:12Amazon Rufus chatbot bypass
- 16:18Healthcare AI file upload attack
- 23:21Automotive AI portal breach
- 26:50AI security dashboard data theft
- 28:16AI hacking skills and techniques
- 30:41Prompt injection taxonomy
- 37:16Chain-of-thought truncation attacks
- 38:40End sequences and prompt confusion
- 40:04Data smuggling through AI models
- 43:15AI supply chain and web app attacks
- 44:39Parseltongue manual AI testing tool
- 49:14AI hacking practice labs
- 51:40Final observations on automation
- 53:25Why prompt injection persists
- 54:31Hacker Prompt CTF results
related talks
01h 14mBreaking the Bot: GenAI Web App Attack Surface & Exploitation
54mA Practical Guide to AI Threats & Mitigation Strategies
20mAI Red Teaming and Automation: Exploring Societal Risks in...
36mAgentic AI Under Attack: Live Demos of Exploits Through Autonomy and Trust
53mHow to Break AI Systems (Before Someone Else Does)
51mPrompt Injection: When Hackers Befriend Your AI
25mPreventing an OWASP Top 10 in the world of AI
30mNavigating the New AI Security Landscape