RL Environments at Scale
summary
Describes an open stack for scaling RL environments: an environments hub plus the verifiers toolkit and a Prime RL trainer that runs thousands of rollouts on hundreds of GPUs. Shows a concrete win: a Qwen 3 4B model rises from ~55% to 89% on a wiki search environment after RL training. Reports hundreds of community-built environments, a 100B+ model Intellect 3 trained on 500 GPUs end-to-end, and tools for SFT, distillation, and eval.
AI-written summary · about
chapters (12)
- 0:00RL environments introduction
- 1:39AI research talent bottleneck
- 3:04Open source as accelerator
- 5:53Environments for AI products
- 7:36Why environments are tools
- 9:01Verifiers toolkit for environments
- 11:27Demo: Wiki Search walkthrough
- 13:11Training small models
- 14:34Scaling to 100B models
- 15:58Community feedback
- 16:41Introducing Lab platform
- 17:43Closing thoughts
related talks
27mScaling an Open Environments Ecosystem for Reinforcement Learning
44mWhy RL Environments Are All You Need for Building Agents
24mverl: An Open Source Large Scale LLM RL Framework for Agentic Tasks
15mRollArt: Disaggregated Multi-Task Agentic RL Training at Scale
39mLet LLMs Wander: Engineering RL Environments
25mVerl: A Flexible and Efficient RL Framework for LLMs
17mRLinf: Flexible and Efficient Large-Scale Reinforcement Learning...
17mScaling Reinforcement Learning