Knowledge Transfer from High-Resource to Low-Resource Programming Languages for Code LLMs
summary
Presents MultiPL-T, a pipeline that generates semi‑synthetic training data by using a Code LLM to synthesize unit tests for 130K vetted Python functions, translating each into five low‑resource targets (Julia, Lua, OCaml, R, Racket), compiling translated tests to filter bad translations, and fine‑tuning models (StarCoderBase, Code Llama, DeepSeek Coder, StarCoder2). Produces tens of thousands of validated target‑language examples, improves pass rates substantially (closes much of the gap to Python), and is runnable on consumer GPUs. MultiPL-T uses StarCoderBase and The Stack.
AI-written summary · about
chapters (17)
- 0:00Knowledge transfer for code LLMs
- 2:06Low-resource language challenges
- 3:53Evaluating with HumanEval
- 4:57Limitations of GitHub data
- 6:01Synthetic data with MultiPLT
- 7:43Filtering translated code
- 8:27Performance gains in low-resource langs
- 9:10Preserving idiomatic code
- 10:12Testing model generalization
- 11:37Impact on StarCoder 2 & Llama
- 12:18Conclusion & future work
- 12:40Q&A: Repository-level translation
- 14:09Q&A: Data scarcity solutions
- 16:18Q&A: Grading model outputs
- 17:01Q&A: Adopting new languages
- 18:05Q&A: Benchmark overfitting risks
- 19:09Q&A: Handling language paradigms
related talks
26mReverse Engineering Using LLMs
32mLinguistic, domain and task adaptation of LLMs using the PLLuM models as an example
41mTeaching Exotic Programming Languages to Large Language Models
33mPractical Evaluation of LLMs for Code
29mLeveraging open-source LLMs for production
01h 02mA hacker's guide to open source LLMs
27mProductionizing Open Source LLMs
02h 52mLow Level Technicals of LLMs