archive – Soumitra Pandit
summary
Explains how PyTorch implements adjoint computational graphs so each operator block carries its backward rule and a forward pass becomes a trace that can be overlaid with a backward pass to compute gradients in one sweep. Claims PyTorch is about 3 million lines of code, uses tensors as the primary data class, and relies on reverse-mode automatic differentiation to scale from thousands to billions of parameters. Describes f.backward style operator blocks, dynamic graphs that change on the fly, and practical space optimizations.
AI-written summary · about
chapters (8)
- 0:00PyTorch and ML basics
- 2:10Functions and composition
- 6:02Function composition in differentiation
- 9:15Operator blocks and backward pass
- 13:52Reverse mode autodiff
- 14:57PyTorch’s adjoint graphs
- 15:18Q&A
- 17:44GPU architecture and tensors
related talks
36mPyTorch: Fast Differentiable Dynamic Graphs in Python
43mExploring Deep Learning Framework PyTorch
12mLightning Talk: Backpropagation-Free Optimization in PyTorch
32mPyTorch Demystified, Why Did I Switch
47mGet to Know PyTorch - A Cutting Edge Open Source AI Framework for Deep Learning
41mReally Deep Neural Networks with PyTorch
32mKeynote: Fireside Chat with Soumith Chintala, Co-Creator of PyTorch & Merve Noyan
26mFrom NumPy to PyTorch, A Story of API Compatibility