KRCORE: A Microsecond-scale RDMA Control Plane for Elastic Computing
summary
KRCORE is an RDMA library that creates full connections in 10μs connection establishment by virtualized kernel-space QPs and retrofitting dynamic connected transport with static transport, keeping only a fixed-sized connection metadata per node. Implemented as a loadable kernel module (≈10k Rust LOC + small C), it cuts RACE Hashing worker bootstrap time by 83% and RDMA transfer latency in Fn by 99%. Trade-offs: DCT-based data path shows lower peak throughput than verbs but KRCORE can dynamically switch to RC to recover performance.
AI-written summary · about
chapters (12)
- 0:00KRCORE and RDMA challenges
- 2:30RDMA bottlenecks analysis
- 4:37KRCORE design with connection pooling
- 7:50Reducing QP overhead
- 10:16Metadata discovery with RDMA
- 11:43Multiplexing QPs for sharing
- 13:30KRCORE implementation overview
- 14:32Evaluation setup
- 15:14Performance comparison with IB verbs
- 16:41Data plane performance
- 17:43Benefits for elastic RDMA
- 18:47Summary and future
related talks
13mRakaia: Scalable In-Kernel Scheduling for TCP-Based RPCs
14mODRP: On-Demand Remote Paging with Programmable RDMA
12mPreview: Networking
22mMetaRoCE: Meta’s RDMA Transport
20mMetaRoCE: Meta’s RDMA Transport
20mRDMA at Cloud Scale: The OCI Experience
13mLive in the Express Lane
17mSRNIC: A Scalable Architecture for RDMA NICs