AI Red Teaming Agent: Azure AI Foundry
summary
Azure’s Red Teaming Agent in the Azure AI Evaluation SDK runs automated scans against applications or models (RAG on Postgres, local OLLAMA, or Azure OpenAI) using composable attack strategies like Caesar/Base64/flip converters and returns filterable results on a hosted dashboard. Small demo scans (n≈160) show zero breaks against a guarded GPT-4o build, but a direct model scan reports 25% violence success and 20% complex-attack success with guardrails off; guardrails reduce but don’t eliminate vulnerabilities. Live demo tooling is brittle but the SDK, agent, and dashboard produce actionable findings.
AI-written summary · about
chapters (6)
- 0:00The challenges of AI safety
- 4:00Red teaming as a team sport
- 5:26Live demo of a red team agent
- 8:45Scaling and automating red team scans
- 14:21A framework for AI risk management
- 17:08Q&A and concluding thoughts
related talks
41mHow Microsoft Approaches AI Red Teaming
27mKeynote: Towards Building Safe and Secure Agentic AI
19mKeynote: Towards Building Safe and Secure Agentic AI
43mSecurity of GenAI & LLMs: A New Era of Ethical Hacking
50mThe ART of Probable: Test with AI, Atomic Red Team, and Threat Metrics
44mRed Teaming AI: Getting Started with PyRIT
43mOperationalize AI responsibly with Azure AI Studio
15mAI Guardrails for Enterprise Deployments