Enabling Efficient GPU Communication over Multiple NICs with FuseLink
summary
FuseLink remaps GPU-side network buffers to relay traffic over high-speed intra-server fabrics (MVLink/MLink) so idle 400G RDMA NICs act as a merged “super NIC” without extra copies or re-registration. Implementation integrates into NCCL on NVIDIA hardware, performs remapping in about 100 microseconds, and uses per-chunk load balancing to avoid NIC contention. Measured peak inter-server bandwidth reaches 212GBps and yields 1.04–2.73× lower first-token latency, 1.3× MoE throughput, and 1.2× recommendation training speedups.
AI-written summary · about
chapters (9)
- 0:00FuseLink and GPU communication
- 2:28Multi-NIC GPU challenges
- 6:05FuseLink design principles
- 8:13Technical challenges and hardware limits
- 10:06Decoupling GPU address and buffer
- 12:31NIC load balancing
- 13:37Implementation and test setup
- 14:42Performance results
- 16:08Future directions and Q&A
related talks
19mFpgaNIC: An FPGA-based Versatile 100Gb SmartNIC for GPUs
14mHigh-throughput and Flexible Host Networking for Accelerated Computing
34mSupercharge Kubeflow Performance on GPU Clusters
23mG-NET: Effective GPU Sharing in NFV Systems
34mCan the Linux networking stack be used with very high speed applications?
28mA Deep Dive on How To Leverage the NVIDIA GB200 for Ultra-Fast...
18mFAST: An Efficient Scheduler for All-to-All GPU Communication
31mHow to capture 100G Ethernet traffic at wire speed to local disk