Transparent MultiNIC Routing for Large AI Models
summary
Meta describes a host-level system that exposes each NIC to containers with per-NIC task IPs, uses an eBPF kernel hook to ECMP outbound flows and installs a lightweight BGP VP injector to advertise a virtual task IP with multiple equal-cost next hops for inbound balancing. Deployment yields concrete wins: a job read latency collapse from 300s to 1s, checkpoint load from 800s to 100s, and roughly 4x throughput on a 4‑NIC server, with NUMA-aware RX steering planned.
AI-written summary · about
chapters (8)
- 0:00AI networking challenges
- 3:21Topology and hardware evolution
- 5:08Analyzing host network bottlenecks
- 7:34Scaling bandwidth solutions
- 11:11Implementing multi-NIC egress
- 13:58Optimizing ingress performance
- 18:45Performance impact results
- 20:25Future advancements
related talks
16mRingLeader: Efficiently Offloading Intra-Server Orchestration to NICs
20mNetKernel: Making Network Stack Part of the Virtualized Infrastructure
39mScalable Multi-Node AI Workloads in Multi-Tenant AI Clouds U...
16mExpress Lane to Efficiency and Reliability: Multi-Dimensional Control in Meta’s Express..
09mLightning Talk: Cloud Native Networking for AI : Strengthen CNI for RDMA
01h 00mCommSec D1 - An Isolated Data Center Security Policy Model Using SmartNICs
28mSafe sharing of the network with eBPF
18mNICA: An Infrastructure for Inline Acceleration of Network Applications