verl: An Open Source Large Scale LLM RL Framework for Agentic Tasks
summary
verl is an open source RL framework for large-scale LLM agentic tasks that claims to let users implement GRPO/PO-style algorithms in a few lines via a HybridFlow single-controller-over-multicontrollers design, and to run distributed generation, rollouts and training with optimizations like offloading and reloading and dynamic resharding. GitHub shows ~8.4K stars, ~200 contributors and 1,000 PRs; 0.4 adds multi‑turn async rollouts, MOE support for DeepSeek v3 and multi-node inference, and reported working integrations with FSDP/FlashAttention.
AI-written summary · about
chapters (6)
related talks
25mVerl: A Flexible and Efficient RL Framework for LLMs
39mAccelerating Open-Source RL and Agentic Inference with vLLM
14mRLlib: Scalable RL for TensorFlow, PyTorch, and Beyond
19mRL Environments at Scale
36mSkyRL: Democratizing RL Training for Researchers
19mTraining Agentic Reasoners
17mRLinf: Flexible and Efficient Large-Scale Reinforcement Learning...
27mSkyRL: A Modular RL Library for LLM Agents