StreamAlloc: A Framework for Analyzing and Transforming CUDA Code to Enable(…)
summary
StreamAlloc implements sync2async as a compiler pass using LLVM/Clang to convert synchronous cudaMemcpy and kernel launches into async calls with automatically chosen streams and synchronizations. The framework uses interprocedural read/write and CRA analyses plus a Data Flow Stream Assignment (DFSA) algorithm, finds 33 async opportunities across 26 benchmarks, transforms IR in milliseconds, and yields geomean speedups of about 1.5x/1.6x/2.0x on P100/A4000/A100 (with peak gains up to 3.2x–4.3x).
AI-written summary · about
chapters (17)
- 0:00StreamAlloc introduction
- 0:22GPU program execution
- 1:25Synchronous CUDA limits
- 2:54Transforming CUDA with async
- 3:57CUDA streams and parallelism
- 5:04Stream optimizations relevance
- 6:08CUDA read-write analysis
- 6:49Run Asynchronously analysis
- 7:53Dependency graph for CUDA
- 8:55Stream assignment challenges
- 10:40Data flow stream assignment
- 12:06DFS stream color assignment
- 13:54Automatic code transformation
- 14:36Performance evaluation
- 15:42Visualizing timelines
- 16:24Summary and future work
- 16:45Q&A
related talks
01h 00mBringing Clang and C++ to GPUs: An Open-Source, CUDA-Compatible GPU C++ Compiler
27mDistributing and Parallelizing Non-canonical Loops
24mMapping Sparse Applications to Accelerated Computing Systems
17mA Sparse Iteration Space Transformation Framework for Sparse Tensor Algebra
19mImprovement and enhancement of LLVM for HPC
01h 02mMemory and Compilers
01h 13mTalk by I-Ting Angelina Lee (Washington University in St. Louis)
01h 02mHydroflow: A Compiler Target for Fast, Correct Distributed Programs