Maximizing Heterogeneous Memory Bandwidth in Multi-Socket Systems
summary
Presents Socket-aware Weighted Interleave implemented in the open-source HMSDK and upstream Linux patches to localize memory interleaving per physical CPU socket by mapping CXL PCI topology to socket membership. Evaluates with Intel MLC, Vector DB workloads and Meta CacheLib, showing peak memory throughput of 1.33 TB/s (≈47% better than prior weighted interleave), UPI link use reduced from 41% to 3%, 99% tail TTFT/latency down to 436 ns (~20% improvement), and elimination of multi-socket remote-access contention that had reduced throughput to ~903 GB/s.
AI-written summary · about
chapters (6)
- 0:00CXL memory bandwidth intro
- 5:21Weighted Interleave limits
- 10:39Socket-aware Interleave design
- 19:10Performance evaluation
- 28:19Results and future plans
- 29:21Q&A
related talks
31mHMSDK: Enhancing Software Support for CXL Memory
26mCXL Memory Tiering for heterogenous computing
37mMeta’s CXL Journey and Learnings in Linux
43mExpanding the Memory Limits with CXL Software Stack
21mPreview: Operating Systems
43mLive in a world with multiple memory types
42mMessing up Your NUMA Topology with CXL
24mOptimizing Storage Performance for 4–5 Million IOPs