How Microsoft Approaches AI Red Teaming
summary
Microsoft describes an operational AI red team that runs over 60 sprints last year, open-sources PyRIT AIRT as its Python automation/orchestration framework, and uses prompt templates, encoders, scorers and targets to scale attacks across text, image and audio. Demonstrations include completing Gandalf levels via autonomous agents, practical XPIA spear‑phish to exfiltrate merger signals, and repeatable scoring to flag jailbreaks and responsible‑AI harms; tooling and findings feed product mitigations.
AI-written summary · about
chapters (11)
- 0:00Microsoft's AI Red Team intro
- 3:04AI Red Teaming principles
- 9:30Three AI red teaming methods
- 13:08Jailbreak and social engineering
- 16:43Prompt injection attacks
- 25:19Modular attack building
- 27:07PyRIT for scalable testing
- 29:36PyRIT demo with Gandalf
- 34:44Community and transparency
- 37:18Industry engagement
- 39:45Q&A
related talks
44mRed Teaming AI: Getting Started with PyRIT
20mAI Red Teaming Agent: Azure AI Foundry
48mRed Teaming AI Systems: The Path, the Prospect and the Perils
27mKeynote: Towards Building Safe and Secure Agentic AI
01h 24mOffensive Cybersecurity by AI Promises and Pitfalls
20mAI Red Teaming and Automation: Exploring Societal Risks in...
50mThe ART of Probable: Test with AI, Atomic Red Team, and Threat Metrics
44mAI Meets Cybersecurity: Boosting Speed Without Losing Control