Beyond Tests: What to Verify in AI-Generated Code
summary
Claim: agents silently inject subtle production bugs that tests miss; verification of invariant-driven framework prevents that. Demo uses a shopping-cart app, shows a race condition fixed by a transaction check detected by an MCP transaction analyzer, a Mongoose query memory bug fixed by adding .lean(), and a combinatorial validation with 2,586 permutations reduced by pre-save hooks. Taxonomy defines scope (global/project/feature), check type (data/business/perf) and remediation (warn/block/rewrite). Tessl Specification Registry is used for spec-backed enforcement.
AI-written summary · about
chapters (7)
- 0:00Introduction and background
- 4:02Three categories of untestable issues
- 12:09Demo: race conditions in transactions
- 16:02Demo: performance with MongoDB
- 17:48Demo: combinatorial state validation
- 20:14Action plan and next steps
- 21:59Summary and Credential.ai
related talks
44mConfidently Wrong: When AI Cannot Catch Its Own Bugs
32mCan We Trust AI-Generated Code?
01h 07mHappiness U-Curve: Navigating the AI Validation Bottleneck with Conformance Testing and(…)
26mThe Trust Problem: Designing Quality Frameworks for AI-Generated Code
13mSpec-Driven Testing for Agents With A Brain the Size of A Planet
01h 01mDiscussion
48mNever Trust a Monkey: The Chasm, the Craft & the Chain of AI-Assisted Code
51mNever Trust a Monkey: The Chasm, the Craft, and the Chain of AI-Assisted Code