Optimizing for GPUs
summary
Concrete measurements and recipes: kernels run in warps of 32 or 64 so choose block sizes that avoid idle threads; grid sizing matters — Titan X favors about global size ≈ 1/4 work, older 750 about 1/16–1/32. GPU allocations and frees block the stream so memory allocations are synchronous and kill throughput if done per-iteration. Branches inside a warp execute both paths so divergence effectively costs the sum of branches. Tiling into shared memory yields multi× speedups versus naive loads and often fixes bad memory layouts.
AI-written summary · about
chapters (6)
- 0:00Introduction
- 2:25Grid vs Blocks
- 4:00Scheduling
- 6:00Block size
- 10:55Synchronization
- 12:40Divergent code
related talks
40mHidden GEMMs: How Optimized Math Libraries Work
01h 06mFoundation of GPU Programming
30mUnderstanding GPUs
08mGPU Programming with Triton Kernels
30mGPU Sharing at CERN: Cutting the Cake Without Losing a Slice!
28mArray Programming on GPUs: Challenges and Opportunities
01h 26mGPU Algorithm Authoring with CUDA Tile
33mWhen GPUs Make Python Slower: Patterns and Pitfalls