How Pytorch Powers Training Inference
summary
Shows how PyTorch stacks combine concrete techniques and numbers: torch Titan composes 3D parallelism, activation recomputation and TorchCompile to run model pretraining; torch tune offers single-file, hackable recipes for adapter, quantization-aware and CPU‑offload workflows so a Llama‑27B needs ~56 GB while Llama‑370B can be adapter‑fine‑tuned on two 4090s; distributed tooling scales Mixture‑of‑Experts to 3,000 GPUs and collaborative work yields 10x faster checkpointing for long runs, plus low‑bit quantization and ExecuteTorch for on‑device inference.
AI-written summary · about
chapters (12)
- 0:00PyTorch and LLMs
- 1:09Large scale training
- 3:54Distributed training primitives
- 6:24TorchTitan pre-training
- 8:11Training large models challenges
- 8:32Fine-tuning LLMs
- 13:10TorchTune fine-tuning
- 15:59Fine-tuning and multimodality
- 16:41On-device inference for LLMs
- 19:30Quantization and sparsity
- 20:57PyTorch inference tools
- 22:44Generative AI future
related talks
16mDe-mystifying PyTorch for ASICs: When (and Why) To Move Your Development To AI A...
11mUnlock the Future of Generative AI: TorchTitan's Latest...
09mTogether Goes Brrr: Threading Research & Production with Torch Compile
23mTorchTPU: Expanding TPU Programmabil...
22mGenAI Training In Production: Software, Hardware & Network Considerations
16mtorchtune: Easy and Accessible Finetuning in Native PyTorch
33mLive Q&A Session 2
25mSponsored Session: Accelerating GenAI Inference: From AWS Deep Learning...