Can We Trust AI-Generated Code?
summary
Claims that LLMs produce correct code only about half the time (slide cites ~50% correctness and ~40% of AI code vulnerable) and that self-verification creates a circular failure motivate a workflow: translate human intent into a machine‑parsable spec, deterministically generate tests from that spec, lock them as read-only tests, then allow models to implement until tests pass. This intent integrity chain recovers undocumented behavior, yields a documented codebase, and uses emerging tools like Kira and SpecKit (Amazon GA) in production experiments since January.
AI-written summary · about
chapters (7)
- 0:00Welcome and introduction
- 1:13AI in software history
- 7:14Nondeterminism in AI code
- 11:26Intent integrity challenge
- 19:55Spec-driven development solution
- 27:06Intent integrity chain process
- 30:20Future of AI and mastery
related talks
51mNever Trust a Monkey: The Chasm, the Craft, and the Chain of AI-Assisted Code
48mNever Trust a Monkey: The Chasm, the Craft & the Chain of AI-Assisted Code
33mShipping Fast, Breaking Silently: Why AI-Generated Code Needs a Second Opinion
09mHow Much Trust Do You Have with LLM-Based Solutions?
34mBuilding Guardrails for AI Coding Systems
26mThe Trust Problem: Designing Quality Frameworks for AI-Generated Code
01h 04mWhat are we thinking in the age of AI?
43mFrom Lambda to LLMs: AI Coding and the Future of Programming