Do LLMs dream of Type Inference?
summary
An LLM-based tool called RBS Goose generates RBS type definitions from Ruby code and shows concrete strengths: it generalizes argument types (make_sound becomes Bad class) and recovers methods defined via define_method, while it still makes RBS grammar mistakes in proctype cases. Current evaluation is ad hoc and manual so the plan named TypeEvalRb creates microbenchmarks plus real projects, builds comparison trees and reports a matches count metric comparing inferred versus expected RBS types.
AI-written summary · about
chapters (6)
- 0:00LLMs and type inference
- 5:24Type systems and inference
- 10:51RBS Goose with LLMs
- 19:19Evaluating type inference
- 25:51Ruby type inference benchmark
- 30:06Summary and future
related talks
37mChecking Your Types: An Overview of Ruby's Type System
32mThe State of Ruby 3 Typing
37mTypes and Ruby Programming Language
53mAn introduction to typed Ruby programming
22mStatic typing with RBS in Ruby
31mBeyond LLMs and Maturing RAG use cases
39mType Checking Ruby Programs with Annotations
22mEnjoy Ruby Programming in IDE and TypeProf