LeapGNN: Accelerating Distributed GNN Training Leveraging Feature-Centric Model Migration
summary
Proposes feature-centric model migration that moves the (small) GNN model to servers holding vertex features, cutting remote feature transfers that dominate communication (about 44% previously). Implements micrograph-based training (one-root subgraphs to boost locality), feature pre-gathering that batches future feature fetches into one transfer, and micrograph merging to reduce time-step synchronizations. Builds LeapGNN and evaluates on a four-server cluster across five datasets and models, achieving 1.1–4.8× speedups across cases and up to 4.2× versus P3.
AI-written summary · about
chapters (7)
- 0:00LeapGNN introduction
- 2:04Distributed training bottlenecks
- 4:37Naive model migration
- 6:48Microraph-based design
- 9:37Feature gathering optimizations
- 11:03Experimental results
- 12:48Summary and future
related talks
16mAccelerating Distributed Graph Learning by Using Collaborative In-Network...
17mBGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessing
14mP3: Distributed Deep Graph Learning at Scale
16mPreview: Graph NNs
29mGraph Neural Networks + LLM on
15mMGG: Accelerating Graph Neural Networks with Fine-Grained Intra-Kernel Communication...
18mTC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUs
19mNeuGraph: Parallel Deep Neural Network Computation on Large Graphs