Robust Generalization in the Era of LLMs: Jailbreaking Attacks and Defenses
summary
Survey of jailbreaking shows concrete attacks and countermeasures: GCG (white‑box gradient‑ascent on prompts) finds adversarial suffixes and hits ~50% on Llama2 and ~33% on GPT‑3.5 but requires hundreds of thousands of queries; PAIR (black‑box attacker‑LLM with chain‑of‑thought) crafts semantically meaningful scenarios and succeeds on GPT and Gemini while failing more on Claude and Llama2. Defenses include randomized smoothing (reduces GCG to <1%) and representation‑based projection (semantic filtering reduces PAIR to ~10–20%); an AdvBench/leaderboard evaluates robust generalization.
AI-written summary · about
chapters (12)
- 0:00Robustness and alignment in LLMs
- 1:05Adversarial machine learning lessons
- 7:47Jailbreaking attacks motivation
- 13:52Threat model for jailbreaking
- 21:26GCG algorithm for jailbreaking
- 29:12White-box jailbreaking limits
- 31:19PAIR black-box method
- 45:41Comparing jailbreaking methods
- 49:12Advanced jailbreaking techniques
- 53:11Defenses against jailbreaking
- 56:11Representation engineering
- 1:00:21Open challenges
related talks
25mWhy LLMs Still Fail: Reasoning, Robustness, and Safety
29mSurvival of the Safest: Automating LLM Defense with Genetic Programming
50mResponsible AI: Adversarial Attacks on LLMs
12mSelfDefend: LLMs Can Defend Themselves against Jailbreaking in...
28mJailbreaking and Securing LLM Apps: Lessons from an online wargame experiment
59mRed Teaming Large Language Models
26mOne guardrail is not enough: Defense-in-depth against LLM jailbreaks
23mSecurity and auditing tools in Large Language Models (LLM)