Efficient Automatic Scheduling of Imaging & Vision Pipelines for the GPU
summary
An autoscheduler generates GPU code from Halide algorithm code by combining a two-phase search that freezes low-cost stages, hierarchical sampling that groups loop-nest structures and evaluates representatives, and memoization of partial schedules to avoid repeated work. A hybrid cost model (small neural net producing weights for hand-designed cost terms) predicts runtimes and guides a beam search over stage-by-stage scheduling. On real imaging pipelines this reduces compile/search time dramatically (average 49x, up to 530x) and finds schedules about 1.7x faster than prior automatic methods, often matching expert-tuned results.
AI-written summary · about
chapters (29)
- 0:00Halide - Separate Algorithm from Schedule
- 1:03Halide Algorithm: 3x3 Box Blur
- 1:30Algorithm + Schedule = Loop Nest
- 1:55Different Schedule = Different Loop Nest
- 2:20Large Space of Schedules
- 2:33Key Challenges
- 3:16Nested Parallelism: Blocks and Threads
- 3:40Prior Work Doesn't Address These Challenges
- 3:57Key Ideas to Address Scalability
- 4:47Basic Overview
- 5:30Overview of Autoscheduler
- 5:40Tree Search Over Space of Schedules
- 5:55Learned Cost Model Guides Search
- 6:08Retrain Cost Model and Repeat
- 7:06Step 1: Where Should blury Be Computed?
- 7:32Step 2: How Should blury be Parallelized?
- 7:43Take the Best Option
- 7:57Take Best Option
- 8:09Run Cost Model
- 8:56Structural Similarity
- 9:43Freezing Unimportant Decisions
- 10:14Memoization
- 10:48Predicting Run Time
- 12:03Cost Model Network: Algorithm Embedding
- 12:14Cost Model Network: Output Coefficients
- 12:24Benchmark and Retrain
- 13:05Results - Freezing and Hierarchical Sampling
- 13:42Results - Applications - Geomean Speedup
- 14:33Conclusion
related talks
57mHalide: A Language for Fast, Portable Computation on Images and Tensors
20mSparseAuto: An Auto-Scheduler for Sparse Tensor Computations Using Recursive Loop Nest(…)
41mHelion's New CuTe DSL and Pallas Backends and How We Built Them Via Agents
15mPipeThreader: Software-Defined Pipelining for Efficient DNN Execution
16mPricing Python Parallelism: A Dynamic Language Cost Model for Heterogeneous Platforms
26mDeepSpeed: Efficient Training Scalability for Deep Learning
18mCompiling Strassen-like Matrix Multiplication Algorithms to Fast CUDA Kernels
24mVerified Tensor-Program Optimization Via High-Level Scheduling Rewrites