GMEM: Generalized OS Memory Management for Accelerators
Talkat Open Source Summit Europe 2023 · on Tuesday, 2023-09-19· at 14:30· Room 3H (3JAUREGIA - Floor 3), Abandoibarra Etorb., 4, 48011 Bilbao, Bizkaia, Spain·
summary
GMEM changes Linux MM to let drivers register hardware-specific mmu functions while the OS handles allocation and mapping, providing a unified virtual address space and transparent memory oversubscription. Implemented on openEuler, it borrows FreeBSD’s VM object for a logical page table and adds async prefetch/hinting. Results: malloc latency in a PyTorch path is ~3x faster, Redis/GCC show <1% overhead, a protein-folding workload increases usable HBM problem size by 25%, and driver MM code shrinks from 30k→30 lines (90% reduction).
AI-written summary · about
chapters (18)
- 0:00GMEM introduction
- 2:12AI memory challenges
- 6:07GMEM design motivation
- 7:34OS managing accelerator memory
- 9:00Current memory limits
- 10:27GMEM core principles
- 11:51Why not Linux HMM
- 13:18Cache-coherent bus limits
- 15:49GMEM implementation
- 17:36GMEM key features
- 18:18Programmability improvements
- 21:54GMEM performance impact
- 22:56Reducing driver bloat
- 23:40Solving memory fragmentation
- 24:23Enhancing large language model inference
- 24:46Memory over-subscription
- 25:29Conclusion and future
- 26:57Q&A
related talks
27mUnified Shared Memory: Friend or Foe? Understanding the Implications of Unified M...
39mBeyond the Baseline: Experimental WebAssembly for High Performance
37mBridging the Architecture Divide: How Hardware Limits Wasm, and How We Can Do Better
29mPerformant Bounds Checking for 64-Bit WebAssembly
18mZVM: Zephyr-based Virtual Machine
26mMemory Safety Preservation for WebAssembly
01h 06mKeynote: Memory of Past and Future – Wasm’s Evolving Model of Store
18mGRANNY: Granular Management of Compute-Intensive Applications in the Cloud