Task-Based Tensor Computations on Modern GPUs
summary
Cypress provides a task-based programming model that maps logical tensor tasks to Hopper GPU resources and lowers to CUDA that competes with handwritten code. It targets TMA and Tensor Cores and models processor and memory hierarchy so the compiler handles asynchronous copies, flags, and synchronization. Implementations include GEMM, batched GEMM and a Flash Attention 3 reimplementation, reaching 0.88x–1.06x cuBLAS and 0.80x–0.98x Flash Attention while removing explicit data movement from application code.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 0:24Talk overview
- 1:09Tensor cores evolution
- 2:33CUDA changes with Hopper
- 3:57Async task parallel challenges
- 5:40Automation in GPU programming
- 7:26Cypress programming model
- 8:29GPUs as hierarchical machines
- 9:53Cypress program structure
- 11:17Mapping to hardware
- 12:20Performance results
- 13:23Flash Attention 3 case study
- 14:46Key takeaways
- 15:29Q&A
related talks
09mLightning Talk: Harnessing NVIDIA Tensor Cores: An Exploration of CUTLASS & OpenAI..
18mCompiling Strassen-like Matrix Multiplication Algorithms to Fast CUDA Kernels
18mTC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUs
01h 18mOptimizing LLM Training and Inference Performance on GPUs
35mComputational model and algorithms for tensor core units
25mLeveraging AI Ecosystem for Portable and Sustainable GPU Kernels in HPC
10mMaximizing Kernel Development Productivity Under Performance Constraints
19mVoltrix: Sparse Matrix-Matrix Multiplication on Tensor Cores with Asynchronous...