Memory Resources in a Heterogeneous World
summary
C++17 std::pmr proves awkward for GPUs because its virtual functions embed raw pointers, so Thrust implements thrust::mr (templated on fancy pointer types) and experiments with upstreams like cudaMalloc and cudaMallocManaged. A practical change is a disjoint pool resource that keeps pool metadata in host memory while device allocations live on the GPU, avoiding metadata/data page thrashing and repeated PCIe transfers; the design ships in Thrust, works for CUDA-managed and non-unified-addressing systems, and also helps multi-NUMA setups.
AI-written summary · about
chapters (26)
- 0:00Introduction
- 1:41Allocators
- 2:40How to use alligators
- 6:26Separation of concerns
- 7:00Alligator
- 9:11Locator
- 10:45Cached Allocator
- 13:20Memory Resources
- 14:54Stateful Alligators
- 17:15Memory Resource
- 22:14CPU vs GPU
- 30:21Pool Resources
- 32:45Inline bookkeeping
- 35:15Unified addressing
- 39:25CUDA Malloc
- 41:13Akane
- 43:03Frost
- 44:17Remer
- 44:56Voidstar
- 45:46CUDA Memory Resource
- 46:20Frost Pointer
- 48:32Bookkeeping
- 51:01Naming
- 51:48Useful
- 52:35Questions
- 59:05Recommendation
related talks
01h 00mBringing Existing Code to CUDA Using constexpr and std::pmr
01h 10mBasic usage of PMRs for better performance
27m“From security to performance to GPU programming..."
20mMemory Harvesting in Multi-GPU Systems with Hierarchical Unified Virtual Memory
43mPromise and Pitfalls of Persistent Memory
51mUnderstanding Allocator Impact on Runtime Performance in C++
29mTowards an Alternative Memory Architecture
49mMaking Allocators Work, Part I