RLinf: Flexible and Efficient Large-Scale Reinforcement Learning...
Talkat 20th USENIX Symposium on Operating Systems Design and Implementation (OSDI '26) · 2026-07-13..07-15·
summary
RLinf is a distributed RL training system that implements macro-to-micro flow transformation to break high-level workflows into optimized execution flows; it ships a worker abstraction with data channels, a priority device lock for auto context switching, and elastic pipelining to vary batch granularity without code changes. A profiling-guided scheduler picks temporal vs spatial scaling, and adaptive zero-copy CUDA IPC handles heterogeneous accelerators. Evaluations on reasoning and embodied RL show 1.07×–2.43× speedup in end-to-end throughput and the project is open-sourced.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 0:43RL scaling background
- 2:07RL workflow challenges
- 4:19Flexibility for efficiency
- 5:22Arlin flow transformation
- 6:25Building macro workflows
- 8:11Workers and groups
- 10:40Fine-grained scheduling
- 11:44Elastic pipelining
- 13:49Guided scheduling policy
- 14:32Adaptive communication
- 15:15Evaluation results
- 15:57Conclusion and open-source details
related talks
25mVerl: A Flexible and Efficient RL Framework for LLMs
26mMSRL: Distributed Reinforcement Learning with Dataflow Fragments
14mRLlib: Scalable RL for TensorFlow, PyTorch, and Beyond
13mOptimizing RLHF Training for Large Language Models with Stage Fusion
24mverl: An Open Source Large Scale LLM RL Framework for Agentic Tasks
32mByteDance's Platform for Reinforcement Learning from Human Feedback
17mScaling Reinforcement Learning
17mReinforcement Learning with Human Feedback in Distributed Systems