GMI-DRL: Empowering Multi-GPU DRL with Adaptive-Grained Parallelism
summary
GMI-DRL introduces a sub-GPU abstraction called GPU Multiplexing Instance (GMI) plus an adaptive Coordinator and a specialized Communicator to map heterogeneous DRL tasks to multi-GPU hardware, adapt instance granularity, and reduce inter-instance traffic. Implementation uses DGX-A100 with PyTorch, NVIDIA gym and NCCL-style communication; experiments show up to 2.34x training throughput and up to 40.8% higher GPU utilization compared to state-of-the-art baselines, and reports better packing and hierarchical gradient reduction strategies.
AI-written summary · about
chapters (9)
- 0:00GMI-DRL research
- 1:03Deep RL basics and GPU optimization
- 3:09GPU utilization patterns
- 4:57Adapting hardware to computation
- 6:24GMI-DRL system design
- 7:47Resource-aware instance mapping
- 10:17GPU communication handling
- 12:45Experiments and results
- 14:50Comparison with techniques
related talks
13mARK: GPU-driven Code Execution for Distributed Deep Learning
32mLeveraging DRA for Conquering ML Workload Challenges on...
17mServing Heterogeneous Machine Learning Models on Multi-GPU Servers...
19mNo GPU Left Behind: Scaling Online LLM Training With...
13mGNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUs
15mMGG: Accelerating Graph Neural Networks with Fine-Grained Intra-Kernel Communication...
19mDynaRL: Flexible and Dynamic Scheduling of Large-Scale Reinforcement Learning Training
31mThe Evolution of Multi-GPU Inference in vLLM