The ART of Probable: Test with AI, Atomic Red Team, and Threat Metrics
summary
Claim: combine generative AI with open-source testing and public reports to produce prioritized, actionable validation tests and detection analytics. Implementation: a ChatGPT Pro Custom GPT seeded with ~20 vendor reports (Red Canary, CrowdStrike, Mandiant, Microsoft, DBIR, etc.), a consolidated Atomic Red Team markdown, and iterative prompt engineering (25–35 revisions). Outcome: the model generates a threat priority list, runnable atomic test commands and sample detection queries, but shows file-size limits and intermittent memory/knowledge-search failures—reliable after tuning, fragile on shared hosted models.
AI-written summary · about
chapters (14)
- 0:00Introduction and context
- 1:00Recap of previous research
- 3:33The challenge of threat modeling
- 6:48Defining your threat model
- 11:24Using AI for threat intel
- 15:04Why AI is effective for security
- 20:01Setting up the custom AI tool
- 24:25Mastering prompt engineering
- 28:24Iterative testing and debugging
- 32:46Breakthrough and results
- 36:32Trivializing test creation
- 39:39Limitations and lessons learned
- 44:00Action plan and conclusion
- 46:12Q&A session
related talks
01h 24mOffensive Cybersecurity by AI Promises and Pitfalls
20mAI Red Teaming Agent: Azure AI Foundry
27mKeynote: Towards Building Safe and Secure Agentic AI
19mKeynote: Towards Building Safe and Secure Agentic AI
25mProtecting AI Infrastructure
40mFantastic Red-Team Attacks and How to Find Them
41mAI-Augmented Testing: How Generative AI, Prompt Eng Turn Testers Into Superheroes
17mAI Safety in the Generative AI Era