Shall We Play A Game? LLM Security in Practice
summary
Workshop shows a hands-on LLM hacking game built on GitHub Codespaces with six levels that reproduce real failures and defenses: system messages containing secrets, regex output filters, input deny lists, and a dual-LLM judge. It reports GitHub Security Lab finding >1,000 OSS vulnerabilities (800+ CVEs) and >10,000 players of earlier seasons. Demonstrations use system message as public knowledge, evasions like base64 encoding and spaced/dashed outputs, and recommends allow-list output validation and least-privilege data access.
AI-written summary · about
chapters (11)
- 0:00Welcome and introduction
- 1:57Game overview and objectives
- 5:11LLM security fundamentals
- 9:57Level one system message flaws
- 24:38Level two user ID exposure
- 39:57Level three output validation bypass
- 1:05:50Level four LLM self-verification bypass
- 1:23:20Level five input filtering evasion
- 1:35:24Level six database and multi-layer attacks
- 1:45:09Key takeaways and defenses summary
- 1:53:03Q&A session
related talks
28mJailbreaking and Securing LLM Apps: Lessons from an online wargame experiment
50mSecuring LLM-Powered Applications: Overcoming Security and Privacy Challenges
09mLLM hacking is underrated
27mSecuring The Gateway And Mitigating Risks In LLM API Integration
44mI Abuse Your LLM Live with 10 Security Vulnerabilities
20mSecurity Threats in Modern LLM Applications
39mLLMs everywhere: The future is now and the past keeps repeating itself
53mBreaching LLM-Powered Applications: Overcoming Security and Privacy Challenges