Prompt Hardening - Secure Code Generation Using AI
summary
A systematic evaluation uses Promptu and an LLM as a judge to run 3,000 code-generation tests across OpenAI models (GPT‑4o mini, GPT‑5 mini with reasoning, GPT‑5.1), comparing 300 prompt variants. Adding the prefix using best practices yields an average +15.3% pass rate; GPT‑5 mini with reasoning reaches ~98% pass but consumes ~5,000 tokens per query. Common failures include hardcoded API keys and DB credentials, missing rate limiting, and absent auth; automated evals produce actionable diffs but not perfect fixes.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 3:57Secure AI code generation journey
- 16:05Research methodology
- 21:29Live demo setup
- 30:27Testing results
- 45:25Future work
- 46:50Prompt security tips
- 48:38Takeaways
- 49:20Q&A
related talks
45mPrompt Hardening - Secure Code Generation Using AI
02h 01mPrompt Engineering and AI Red Teaming
24mBetter Prompting with Pseudocode
01h 08mIn Prompts We Trust
44mThe Science of Effective Prompting
59mUsing AI for Secure Code Creation: Enhancing Software Security
43mBecome a 10x developer by harnessing the power of Generative AI
52mUsing GenAI on and inside your code, what could possibly go wrong?