Welder: Scheduling Deep Learning Memory Access via Tile-graph
summary
Welder is a deep-learning compiler that builds a tile-graph abstraction to connect operators at register/shared-memory levels and schedule tiling across operators, using tile propagation and a tile traffic-based cost model to minimize cross-layer memory movement. It fuses operators into larger kernels, pulls high-performance microkernels from CUTLASS, and handles arbitrarily large inputs by combining device and host memory. Evaluation across 10 models reports geometric-mean speedups (≈1.4× vs TensorRT, ≈2× vs some runtimes, ≈1.5× vs ONNX Runtime) and about a 40% gain on RTX 3090.
AI-written summary · about
chapters (12)
- 0:00Welder system introduction
- 0:50Memory bottlenecks in DL inference
- 2:17Limitations of single-operator tiling
- 3:02Tile-graph for inter-operator reuse
- 4:06Welder architecture and workflow
- 6:54Tile shape optimization
- 7:15Graph connecting and scheduling phases
- 9:02Code generation with kernels
- 10:09Compilation speed optimizations
- 10:30Evaluation setup and results
- 11:57Ablation study comparisons
- 12:58Conclusion and future trends
related talks
01h 00mThe Shape of Things to Come
31mBuilding Inference Workflows with Tile Languages
01h 26mGPU Algorithm Authoring with CUDA Tile
19mOptimizing for GPUs
11mQiMeng-Xpiler: Transcompiling Tensor Programs for Deep Learning Systems with a...
17mTessera: A Holistic Pipeline Parallelism Framework for Trillion-Parameter Heterogeneous
25m"Rust for AI & Accelerated Computing"
28mThe Future Is Tiled: Using CuTile & TileIR To Write Portable, High-performance GPU...