Novice Type Error Diagnosis with Natural Language Models
summary
Uses pretrained Transformer language models (BERT/CodeBERT family) to predict token-level type error locations by treating code as token sequences with token-level binary labels, training end-to-end with no hand-crafted features. Achieves 62% accuracy on novice OCaml errors, a 11% improvement over prior data-driven baselines, and uses structural probes to explain model differences. Evaluation uses GitHub and student homework corpora; future work adds AST-aware models such as graph neural networks.
AI-written summary · about
chapters (12)
- 0:00Type error diagnosis intro
- 2:26Why compilers fail
- 3:51Natural language model approach
- 7:05How BERT and GPT work
- 10:40Model accuracy metrics
- 12:25Type error prediction metrics
- 14:58Model performance comparison
- 16:43Model size and transfer learning
- 18:50Generalization to unseen data
- 20:17Advantages and limitations
- 22:23Future work and program structure
- 23:10Q&A
related talks
17mNatural Language Processing in Automation
18mEvaluating the effectiveness of Deep Learning Models for Foundational Program Analysis(…)
01h 00mInside GPT – Large Language Models Demystified
45mHow multilingual is your NLP model?
11mCollaborative Natural Language Inference
29mHallucination by Design: How Embedding Models Misunderstand Language
36mIt's Alive! Machine Learning Writes Your Code! 🧠🔮
40mPanel discussion "NLP"