[Full Workshop] Reinforcement Learning, Kernels, Reasoning, Quantization & Agents
summary
Claims that RL can replace stages of SFT, plus a concrete algorithmic tweak: GRPO removes the value model and uses rollout statistics as the baseline, letting rollout size 4 and temperature ~1.2 find rare successful trajectories in hours on a free Colab. The workshop shows reproducible notebooks via Unsloth, LoRA tuning from ~118 examples, and practical knobs (min_p 0.1, gradient accumulation) that work. Quantization results report DeepSeek-R1 compressed from ~730GB to ~140GB and usable at 1.58-bits with ~1% accuracy loss; Triton/VLM and torch.compile yield ~50–70% memory savings.
AI-written summary · about
chapters (12)
- 0:00Introduction and Unsloth's Contributions
- 3:25The Evolution of Large Language Models (LLMs)
- 9:47LLM Training Stages and Yann LeCun's Cake Analogy
- 16:56Agents and Reinforcement Learning Principles
- 23:17PPO and the Introduction of GRPO
- 48:12Reward Model vs. Reward Function
- 51:22The Math Behind the Reinforce Algorithm
- 1:08:50PPO Formula Breakdown
- 1:16:29GRPO Deep Dive
- 2:00:20Practical Implementation and Demo with Unsloth
- 2:33:07Quantization and the Future of GPUs
- 2:41:59Conclusion and Call to Action
related talks
18mMaximizing Luck in Reinforcement Learning
01h 31mWorkshop on developing Reinforcement Learning
01h 17mTraining intelligent game agents using deep reinforcement learning
01h 11mFoundations of RL
14mClosing the Train-Inference Divide for Asynchronous RL at Scale
20mEverything I Learned Training Frontier Small Models
42mPost-Training Small Language Models: the adventures of a practitioner
01h 05mFoundations of RL