NeuGraph: Parallel Deep Neural Network Computation on Large Graphs
Talkat USENIX ATC '19 · on Thursday, 2019-07-11· at 11:55· USENIX ATC Track II: Grand Ballroom VII–IX·
summary
NeuGraph implements a TensorFlow-layer that fuses graph and dataflow models, adding trunk-based partitioning and a column-by-column scheduler, custom GPU kernels and GPU-to-GPU streaming to avoid host bottlenecks. It translates GNN stages into trunk programs with auto-differentiation, uses selective scheduling to avoid loading whole sparse tensors, and supports graphs larger than GPU memory. On benchmarks it reports up to ~47× speedup over raw TensorFlow and up to ~19× over DGL, with near-linear multi-GPU scaling.
AI-written summary · about
chapters (9)
- 0:00Introduction to GNNs
- 1:36Limitations of current systems
- 4:11Proposed graph model
- 5:01Implementation stages
- 6:58Scalability and scheduling
- 9:45Multi-GPU optimization
- 11:50Evaluation and results
- 14:13Conclusion and Q&A
- 15:27Deep dive into architecture
related talks
18mTC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUs
15mEffectively Scheduling Computational Graphs of Deep Neural Networks toward Their...
16mPreview: Graph NNs
40mRun Large-Scale Graph Analytics Using The Most Popular Graph Analytics Library Available
15mMGG: Accelerating Graph Neural Networks with Fine-Grained Intra-Kernel Communication...
32mTowards new deep learning abstractions on top of exist frameworks
13mGNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUs
39mUsing Graph Neural Networks to Embrace the Dependency Within Your Data