Responsible AI: Adversarial Attacks on LLMs
summary
Researchers generate a universal adversarial suffix by running a white-box gradient + discrete optimization across 15–25 seed prompts to force affirmative responses; those suffixes take 12–24 hours to find and transfer to proprietary chatbots so they answer harmful requests. Evaluation uses about 500 harmful behaviors and shows frequent jailbreaks of GPT-3.5 and sporadic breaks of larger models. Mitigations that work include adversarially fine‑tuning models and continuous adversarial evaluation; simple suffix blacklists and single-layer guard AIs are fragile.
AI-written summary · about
chapters (10)
- 0:00Responsible AI introduction
- 1:26Aligned model attacks
- 10:20Universal adversarial suffixes
- 14:02Tool-enabled AI risks
- 17:36Confused deputy problem
- 20:48Adversarial examples overview
- 24:43Robust training defenses
- 27:45Secure AI deployment
- 36:27Fine-tuning risks
- 42:40Q&A
related talks
01h 04mRobust Generalization in the Era of LLMs: Jailbreaking Attacks and Defenses
24mSecuring Language Models Against Prompt Injection
18mDefending the unknown using AI and cybersecurity
26mWhen AI Gets Manipulated: Security Risks in LLM Applications
26mHow Bad Actors AbUse LLMs The Mechanisms That Enable Misuse and What We Can Do About
37mHow to Break AI Systems (Before Someone Else Does)
53mBypassing LLM Defenses with Adversarial Suffixes and AutoDAN
43mSecurity of GenAI & LLMs: A New Era of Ethical Hacking