DistRS: Disaggregated Reward Service for RLVR with Batch-Level Constraint
summary
DistRS is a disaggregated reward service that separates reward computation from training, implements a microservice pipeline with per-stage worker pools, and uses a history-based simulation-based search scaler plus a timeout-aware reallocation rule to handle long-running requests. An earliest-batch-first (EBF) priority scheduler reduces head-of-line blocking. Deployed on real CUDA code-generation RLVR traces with 4‑CPU compilers and 1‑GPU runners, DistRS cuts GPU time up to 3.79× and CPU time 1.98× while adding only ~0.62s extra delay.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:46Verifiable reward computation
- 3:34Request-level resource disaggregation
- 6:06Workload characteristics
- 7:56Batch-level constraint
- 8:39DistRS architecture
- 9:44Long-running requests
- 11:30Multi-tenant scaling
- 12:54Evaluation setup
- 13:36Performance results
- 14:20Request scheduling
- 15:02Conclusion
related talks
19mDynaRL: Flexible and Dynamic Scheduling of Large-Scale Reinforcement Learning Training
26mMSRL: Distributed Reinforcement Learning with Dataflow Fragments
17mScaling Reinforcement Learning
15mRollArt: Disaggregated Multi-Task Agentic RL Training at Scale
17mRLinf: Flexible and Efficient Large-Scale Reinforcement Learning...
32mByteDance's Platform for Reinforcement Learning from Human Feedback
20mEfficient Reinforcement Learning
12mElastic Resource Sharing for Distributed Deep Learning