Really Deep Neural Networks with PyTorch
summary
Shows concrete claims and code not buzzwords demonstrates PyTorch's tape-based automatic differentiation and GPU tensors and compares numpy implementations to show gradient flow differences gives a small example where switching activations boosts accuracy to ~92 percent explains why vanishing gradients block deep nets and how skip connections (used in ResNet-152 and networks with up to a thousand layers) restore training stability notes DenseNet concatenation and real use cases in chemical genetic profiling and autonomous driving
AI-written summary · about
chapters (41)
- 0:00Introduction
- 0:35What is this talk about
- 1:55The first neuron
- 3:45Teaser
- 4:40What is Deep Learning
- 6:01Classification
- 6:29Reinforcements
- 7:46Small Network
- 8:23Simple Setup
- 8:59Summary
- 11:47Loss Functions
- 12:17Accuracy
- 13:02partitioning
- 13:15architecture activation
- 13:27gradient and sum
- 13:37gradient absolute value
- 14:34print absolute value
- 15:07vanishing gradient problem
- 15:19vanishing gradient
- 16:59sigmoid
- 17:54problem
- 18:24solution
- 18:47auto differentiation
- 19:08PyTorch
- 22:06Computation graph
- 22:44Parallelization
- 23:08Model Paracin
- 23:25Dynamic Computation
- 24:30Static Computation
- 25:17Conditions
- 26:12Sequential Layers
- 26:27Last Function
- 27:22Stacking Layers
- 27:45Stacking Up
- 29:53Shortcut Connection
- 30:57PyTorch Implementation
- 31:41Imagenet Competition
- 32:21Image Classification
- 32:39Dense
- 34:16QA
- 36:24Gross number of layers
related talks
27mDemystifying Deep Neural Networks
01h 26mDeep Neural Networks with PyTorch
01h 05mDeep Learning with PyTorch
49mDynamic Neural Networks using PyTorch
30mAnyone can Build Great Deep Learning Applications - Deep Numpy
28mDemystifying Deep Neural Nets
32mPyTorch Demystified, Why Did I Switch
51mDeep Learning - What is it and What It Can Do For You