DelayRepay: Delayed Execution for Kernel Fusion in Python
summary
DelayRepay is a drop-in NumPy replacement that defers NumPy operations to build a delayed execution graph, fuses those nodes into single CUDA kernels via CuPy, and emits SSA CUDA with CSE and scalar partial evaluation. It achieves a 24× geometric-mean kernel speedup (CuPy 15×, Bohrium 11×) and a 377× max on one benchmark, and a cold end-to-end 1.25× mean speedup versus CuPy/Bohrium slowdowns; one pathological case with many random constants produces thousands of kernels and hurts performance.
AI-written summary · about
chapters (15)
- 0:00Intro
- 0:22Executive Summary
- 1:20Accelerators
- 2:09GPU Python
- 3:20Kernel Fusion
- 4:34Our Solution: Delay Repay
- 4:53Drop-in Numpy Replacement
- 5:57Delayed Evaluation
- 6:34Tracing Numpy
- 8:44GPU Code Generation
- 12:04Results Warm - Speedup
- 12:52Results Cold - Speedup
- 13:24Memory Usage
- 13:55Discussion
- 14:40Future Work
related talks
16mIntrepydd: Performance, Productivity, and Portability for Data Science Application Kernels
39mAccelerated Data Science Libraries
30mCuPy: A NumPy-compatible Library for GPU
02h 55mGPU Accelerated Python w: CUDA
30mCUDA in Your Python: Effective Parallel Programming on the GPU
34mAccelerating GPU Algorithms in pure Python
25mScale Smarter, Not Harder, with cuPyNumeric.
02h 28mGPU Accelerated Python