Red Teaming AI: Getting Started with PyRIT
summary
PyRIT claims to automate and semi‑automate AI red teaming by generating attacks with an adversarial fine‑tuned LLM and scoring outputs with LM‑based judges, while retaining a human‑in‑the‑loop for difficult labels. The toolkit ships with 30+ converters, supports OpenAI‑compatible endpoints and Playwright web testing, logs everything to a results DB (SQLite or Azure SQL), is open source with ~100 contributors, and proves useful in practice though scoring accuracy and scaling remain the main limits.
AI-written summary · about
chapters (17)
- 0:00AI red teaming and PyRIT intro
- 1:26Generative AI needs red teaming
- 5:21AI vs traditional cybersecurity
- 8:51Diverse AI red team building
- 11:59Scaling red teaming with PyRIT
- 13:23PyRIT converters and attacks
- 18:18Targeting AI systems with PyRIT
- 19:01Scoring attacks in PyRIT
- 21:08Automating attacks with adversarial AI
- 24:21Predefined red teaming scenarios
- 26:50Human-in-the-loop with PyRIT
- 29:20Challenges and future of AI red teaming
- 32:08Q&A: LLM permissiveness
- 34:57Q&A: Red vs blue for small teams
- 37:28Q&A: Custom languages and jailbreaks
- 39:15Q&A: Harm categories and attacks
- 42:47Q&A: Measuring bias in AI
related talks
41mHow Microsoft Approaches AI Red Teaming
20mAI Red Teaming Agent: Azure AI Foundry
48mRed Teaming AI Systems: The Path, the Prospect and the Perils
50mThe ART of Probable: Test with AI, Atomic Red Team, and Threat Metrics
34mLLM Security 101 - An Introduction to AI Red Teaming
33mSecuring Agentic AI With PyTorch: Threat Modeling & LLM Red Teaming in Practice
25mProtecting AI Infrastructure
20mAI Red Teaming and Automation: Exploring Societal Risks in...