Lightning Talk: Dynamic Shape Recompilations
summary
Explains when recompilations from dynamic shapes are worth the cost with concrete numbers: a 20 ms kernel win invoked 100 times nets 2 s but loses to a 5 s compile, while a 5 ms win invoked 8,000 times pays off; unpredictable guards (example: fast path only if batch size divisible by 8) cause recompiles as shapes like 5,6,7 then 8 trigger new kernels; distributed runs (100 GPUs) stall for single-GPU recompiles and can hit >10 minute nickel timeouts. Covers mark_dynamic usage, mark_unbacked as an error-on-specialization guard, and guarded recompilation trade-offs.
AI-written summary · about
related talks
36mUnderstanding Recomposition Performance Pitfalls
40mLightning Talks
10mLightning Talk: Resource Fairness and Utilization for Heterogeneous Batch/M...
43mFine-grained everything
01h 04mThe Magic Behind Optimizing Compilers: Static Program Analysis
50mBehind the Scenes of the LabVIEW Compiler
26mDon’t Let Compilation Set the Length of your Coffee Break
34mCompose Performance: Hunting for unnecessary recompositions