Building Mini Minds Our Adventures in Teaching Small LLMs to Reason
summary
Reports training 3B and 8B LLaMa models with PPO reinforcement learning on rented GPUs, using ~10K synthetic Dutch station-name stories to seed vocabulary, paying only a few hundred dollars while reporting melted GPUs and messy runs. Small models climb from ~15% to roughly 50% on the anagram game with a ~15% validation uplift, producing long self-verifying outputs, loops, phrases like "just kidding," and humanlike frustrations before finally reaching reproducible “aha!” moments that demonstrate verification and correction working at small scale.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:27Rise of reasoning models
- 3:56How LLMs work
- 6:02Need for reasoning models
- 7:48LLM performance plateau
- 9:56What makes reasoning models different
- 11:01Reinforcement learning training
- 12:27DeepSeek R1 breakthrough
- 14:55Dutch station anagram demo
- 17:25Small models on a budget
- 18:48Humanlike reasoning observed
- 20:54Key insights from experiments
- 21:57Reasoning future of AI?
- 23:24Generalizing beyond math?
- 24:47Call to action
- 26:12Q&A
related talks
44mHow to build your own Tiny Language Model from scratch
19mTraining Agentic Reasoners
28mFine-Tuning Small Language Models with Python
42mPost-Training Small Language Models: the adventures of a practitioner
10mWhat goes into the making of LLMs
15mAgentic Reasonings with Reinforcement Learning
39mLet LLMs Wander: Engineering RL Environments
25mWhy LLMs Still Fail: Reasoning, Robustness, and Safety