PyTorch: Fast Differentiable Dynamic Graphs in Python
summary
PyTorch claims fast, dynamic computation by combining an automatic-differentiation engine with a mixed Python and C/C++ tensor library that runs on GPUs. It implements a tracing JIT with operator fusion to turn repeated Python code into optimized C++ traces, yielding roughly 30–40% speedups for fused chains. Tensors share memory with numpy for zero-copy interop, CUDA transfers are explicit, and the tracer currently struggles with arbitrary Python control flow.
AI-written summary · about
chapters (18)
- 0:00Intro
- 0:40Overview of the talk
- 2:34Machine Translation
- 3:38Adversarial Networks
- 3:54Adversarial Nets
- 7:03Chained Together
- 7:44Trained with Gradient Descent
- 9:56Computation Graph Toolkits Declarative Toolkits
- 12:01Imperative Toolkits
- 16:50Seamless GPU Tensors
- 19:25Neural Networks
- 22:24Python is slow
- 25:24Types of typical operators
- 26:37Add - Mul A simple use-case
- 27:52High-end GPUs have faster memory
- 28:52GPUs like parallelizable problems
- 30:07Compilation benefits
- 31:22Tracing JIT
related talks
43mExploring Deep Learning Framework PyTorch
32mPyTorch Demystified, Why Did I Switch
19mDIY PyTorch - How to Learn Deeply
19marchive – Soumitra Pandit
24mAdvancements in High-Performance AI/ML through PyTorch's Python Compiler
49mDynamic Neural Networks using PyTorch
13mPyTorch 2.0
12mLightning Talk: CUDAGraph in a Partial Graph World