Survival of the Safest: Automating LLM Defense with Genetic Programming
summary
An open-source framework performs systematic red‑teaming against 25 prompt‑based attack types and uses genetic programming to evolve small editing rules that insert anchors and special characters into user prompts, keeping models unchanged. Trained on ~1,000 prompts and evaluated on 33 models and ~22 jailbreak variants, the evolved rules transfer across models and improve safety for 27/33 models (about 73% success) while degrading a few small models; evolution prioritizes a fitness that balances attack reduction and minimal edits so benign utility stays high. No model weights are changed.
AI-written summary · about
chapters (8)
- 0:00LLM jailbreak defense intro
- 1:21Jailbreak attacks today
- 3:12Jailbreak attack types
- 8:34Defense strategies
- 13:34Genetic programming for prompts
- 19:32Evolutionary optimization
- 20:57Testing results and safety
- 23:43Q&A
related talks
59mRed Teaming Large Language Models
01h 04mRobust Generalization in the Era of LLMs: Jailbreaking Attacks and Defenses
21mProtect your generative AI applications against jailbreaks
57mKeynote: Using LLMs and generative AI to fix software vulnerabilities
45mPrompt Hardening - Secure Code Generation Using AI
50mResponsible AI: Adversarial Attacks on LLMs
26mOne guardrail is not enough: Defense-in-depth against LLM jailbreaks
30mBattling the Machine: Cutting-Edge Attacks and Defenses in Generative AI