Conspirator: SmartNIC-Aided Control Plane for Distributed ML Workloads
summary
Conspirator is a SmartNIC-based control plane that offloads data movement and scheduling to an off-path BlueField SmartNIC, using DMA to push client tensors directly into A100 GPU memory via NVIDIA DOA and a small CUDA extension, with only one line of Python change to existing code. Scheduling is framed as a MILP and solved by a fast heuristic (H-stick). Evaluation shows 15% end-to-end completion time reduction vs RDMA alternatives, 17% cost and 44% power improvements, and saves 33% GPU hours versus naive GPU-sharing.
AI-written summary · about
chapters (14)
- 0:00Conspirator SmartNIC project
- 1:46CPU bottlenecks in ML training
- 3:16RDMA and GPU direct limits
- 5:01Accelerator scheduling and GPU sharing
- 7:11SmartNIC solution overview
- 8:36Conspirator system architecture
- 10:25Scheduling model and implementation
- 12:34System implementation and code
- 12:54Evaluation setup and testbed
- 13:37Performance results and batch size
- 14:45CPU load and data speed
- 15:30Cost and power efficiency
- 16:13Scheduling efficiency and savings
- 16:55Conclusion
related talks
17mWhite-Boxing RDMA with Packet-Granular Software Control
15mCharacterizing Off-path SmartNIC for Accelerating Distributed Systems
01h 00mCommSec D1 - An Isolated Data Center Security Policy Model Using SmartNICs
17mSmartNIC-Enabled Live Migration for Storage-Optimized VMs with PYROCUMULUS
17mBarre: Empowering Simplified and Versatile Programmable Congestion Control in...
16mRingLeader: Efficiently Offloading Intra-Server Orchestration to NICs
40mOptimize Your AI Cloud Infrastructure: A Hardware Perspective
20mScheduler and Sharding Considerations for Network Efficiency