Effectively Scheduling Computational Graphs of Deep Neural Networks toward Their...
summary
A new scheduler partitions DNN computation graphs into coarse-grained subgraphs that fuse operators into larger kernels to keep tensors in local buffers and turn many off-core transfers into on-core data exchanges. It also packs imbalanced memory use across stages and performs across-layer instruction scheduling, mapping instances to clusters and LP buffers. On a DSA platform it outperforms TVM and AStitch by 11.15× over TVM and 6.16× over AStitch, and GPU kernels beat CUTLASS by 1.06× and 1.23× with and without fusion.
AI-written summary · about
chapters (7)
- 0:00Scheduling DNN graphs
- 2:55GraphTrop scheduler design
- 5:07Subgraph instances scheduling
- 9:24Loop fusion for kernels
- 11:10Performance comparison
- 12:33Ablation study
- 14:40Summary and future work
related talks
19mNeuGraph: Parallel Deep Neural Network Computation on Large Graphs
23mDecentralized Application-Level Adaptive Scheduling for Multi-Instance DNNs on...
10mFair Scheduling for Deep Learning Workloads in Kubernetes
25mDaydream: Accurately Estimating the Efficacy of Optimizations for DNN Training
24mTiresias: A GPU Cluster Manager for Distributed Deep Learning
20mScheduler and Sharding Considerations for Network Efficiency
13mGNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUs
18mTC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUs