Hostping: Diagnosing Intra-host Network Bottlenecks in RDMA Servers
summary
Hostping runs loopback tests between RNICs and endpoints to measure intra-host latency and bandwidth and uses baseline comparison plus a binary network tomography–style inference to pinpoint faulty links. It diagnoses hardware faults (PCIe/PSI link failures, memory-channel and CPU-port faults), misconfigurations (ATS/ACS), and contention (overloaded UPL), and adapts tests when servers are busy. Deployed on 300+ A100 servers, it reveals six unseen bottlenecks and shows examples where PCIe latency rising from ~1µs to 2.4µs halves RDMA bandwidth.
AI-written summary · about
chapters (8)
- 0:00Intro
- 0:29Background & Motivation
- 4:11Understanding: Symptoms of Intra-host Bottlenecks
- 6:13Hostping Design: Core Idea
- 7:16Hostping Design: Bottleneck Diagnosis (Idle Status)
- 10:58Hostping Design: Bottleneck Diagnosis (Busy Status)
- 12:45Evaluation
- 14:34Conclusion
related talks
15mCollie: Finding Performance Anomalies in RDMA Subsystems
02h 53mPreview Sessions
12mPreview: Networking
01h 24mPreview Session
47mThe World of 100G Networking
23mMulti-Path Transport for RDMA in Datacenters
19mAI/ML Networking Challenges: The Fast and the Finicky!
01mThe Future of AI Networking Starts Here
