Scheduled Partial-Credit RL for Reliable Code Generation with Small Language Models (WIP)
summary
A reliability-first pipeline uses LoRA supervised finetuning (≈60M of 1.3B params) then PPO with a joint reward: staged functional partial credits (syntax validity, crash-free execution, incremental test-case credit) plus a static-analysis security guardrail (bandit SSD2). Evaluated on 100 stdin-style APPS+ prompts with DeepSeek-Coder-1.3B, a partial-credit curriculum raises syntax-valid output to 63% syntax validity and yields ≥1 test pass on 9% of prompts; binary PPO collapses below SFT.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:39Motivation for small LMs
- 3:27Syntax validity issues
- 4:29Baseline SFT results
- 5:35Dual-reward RL
- 7:02Sparse rewards problem
- 8:29Future work
- 9:33Key takeaways
- 9:57Q&A on language choice
- 11:21Q&A on model limits
- 12:45Closing remarks
related talks
45mPrompt Hardening - Secure Code Generation Using AI
19mRL for Autonomous Coding
24mRecursive Coding Agents
40mCode-First LLMOps from prototype to production with GenAI tools
12mLanguage-Model Probabilistic Programming for Improving Autoformalization via Cycle(…)
01h 02mThe Key Ingredients of Optimizing Test-Time Compute and What's Still Missing
29mOne-Click Code Fix: Securing Code Using AI
58mPrompt Hardening - Secure Code Generation Using AI