PyTorch Performance
summary
PyTorch reports concrete speedups and tools: tf32 on Ampere uses 32-bit storage with 10-bit mantissa reads and shows about 3x speedup on huge phase roberto model and ~2x on steroid convents such as resnext versus fp32; automatic mixed precision (AMP) (from Apex) provides autocast, loss scaling and minimal code changes to get better fp16 throughput. Channels‑last memory layout yields roughly 20% gain, foreach APIs speed optimizers by ~3–6x, and benchmarking utilities (Timer/Compare) standardize measurements.
AI-written summary · about
chapters (4)
related talks
05mApex
06m⚡ Speedy but Sturdy PyTorch Optimizers
27mAccelerating Generative AI
31mPyTorch 2.1 - New Developments
16mtorchtune: Easy and Accessible Finetuning in Native PyTorch
25mmxfp8, mxfp4, nvfp4 formats and applications in PyTorch
24mAdvancements in High-Performance AI/ML through PyTorch's Python Compiler
12mLightning Talk: Accelerating Inference on CPU with Torch.Compile