A Profile-Driven Dynamic Optimization Framework for GPU Applications
summary
A framework called Leo uses runtime profiling and dynamic instrumentation to choose GPU implementations automatically instead of manual tuning, mapping high-level Dandelion programs to multiple kernel variants and runtime parameters and switching based on observed hotspots and memory/access patterns. Evaluation on benchmarks shows 1.12 to 27x kernel speedups and 7–40% end-to-end gains, with the system actually generating and measuring multiple code versions, picking different decomposition and launch parameter combinations per workload.
AI-written summary · about
chapters (7)
- 0:00Introduction to framework
- 4:42Profile-driven optimization basics
- 8:20Dynamic instrumentation techniques
- 13:03GPU performance measurement
- 17:31Case studies and results
- 22:43Optimizing GPU applications
- 25:44Q&A and summary
related talks
12mContinuous Profiling for GPUs
20mUnlocking Performance: Harnessing LLMs To Streamline GPU Kernel Development in...
24mInside MoE Optimization: A Profiler-Guided Tour of torch.compile and vLLM
18mKernelAgent: Hardware-Guided GPU Kernel Optimization Via Multi-Agen...
11mAccelerated Software for a Post-Moore World
26mVLLM-Helion: SOTA LLM Performance by Advanced Autotuning and Fine-grained Dispatching
13mARK: GPU-driven Code Execution for Distributed Deep Learning
08mLightning Talk: GPU.x - Dynamic GPU Fraction and Sharing on ML Mainstream Framework