cudaFlow: Modern C++ Programming Model for GPU Task Graph Parallelism
summary
A C++ library called cudaFlow wraps NVIDIA CUDA Graph to build and offload large GPU task graphs with thousands of nodes, provides an automatic cudaFlow capture transformation to assign streams/events, and supports incremental graph updates. Benchmarks use up to four RTX 2080 Ti GPUs, show up to about 2× speedup over prior systems on a 65k‑neuron 1920‑layer inference case, and drop graph management overhead from ~70% to ~5%, so the implementation scales and meaningfully reduces CPU/GPU scheduling cost.
AI-written summary · about
chapters (29)
- 0:00Introduction
- 0:33Cudaflow motivation
- 1:21GPU computing benefits
- 2:59Cuda programming models
- 3:51AXPY example
- 5:13Complex GPU workloads
- 7:08Stream-based execution
- 8:50Stream limitations
- 10:57CUDA graph model
- 13:00Explicit vs implicit
- 15:16Cudaflow overview
- 17:33Cudaflow programming
- 20:15Cudaflow capturer
- 22:21Design philosophy
- 24:09API categories
- 27:54API restrictions
- 29:33Taskflow integration
- 31:05Performance tips
- 34:47Capturer details
- 37:02Transformation goals
- 39:00Transformation algorithm
- 40:25Algorithm example
- 43:36ML case study
- 46:12Capturer performance
- 47:48Update performance
- 48:46Circuit simulation
- 50:30Conclusion
- 52:31Q&A
- 1:01:08Wrap up
related talks
05mFast Parallel Programming using Modern C++
01h 00mC++ on GPUs done right?
01h 06mAsynchronous Computation in C++
01h 02mC++ Performance Portability - A Decade of Lessons Learned
01h 02mTaro: Task Graph-Based Asynchronous Programming Using C++ Coroutine
31mGPU Performance Portability Using Standard C++ with SYCL
37mC++ for multi-accelerator, multi-vendor systems
39mFearless Concurrency on the GPU