Type-Constrained Code Generation with Language Models
summary
Introduces a type-constrained decoding method that integrates an incremental type checker into token-level LLM decoding using prefix automata plus a search over inhabitable types to ensure generated TypeScript is well typed in one forward pass. Implements finite bounds on the type-reachability graph, supports many TypeScript features, and ships reusable artifacts. Evaluated on HumanEval and MBPP with models up to ~32B, cutting compilation errors by over half (50–60%+) and producing several percentage points of functional-correctness improvement, especially large (≈80%) gains in repair mode.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:06LLM code vs type systems
- 2:29Type systems in decoding
- 3:34Incremental type checking demo
- 6:45Automaton for type checker
- 8:51Visualizing automaton
- 11:41Type inhabitation problem
- 13:29Benefits and manual construction
- 14:13Evaluation setup
- 15:59Results on correctness
- 17:02Limitations and future support
- 18:05Conclusion and Q&A
related talks
20mStatically Contextualizing Large Language Models with Typed Holes
01h 22mBeyond Decoding: Meta-Generation Algorithms for Large Language Models
11mTypeScript was built for agent factories
36mIt's Alive! Machine Learning Writes Your Code! 🧠🔮
53mSemantics-based program manipulation in the age of LLMs
34mIt's Alive! Machine Learning Writes Your Code! 🧠
26mChoosing TypeScript Matters More Than Ever
37mGPT-3-Powered Type Error Debugging: Investigating the Use of Large Language Models...