Tutorial on Deep Learning I
summary
Claims that modern deep nets win only with very large datasets and practical tricks: hierarchical representations work well for vision, language and even medical tasks; empirical elements are ReLU activations and training with SGD plus momentum, plus regularizers like dropout with p=0.5 and batch normalization per-mini-batch. Examples shown: unsupervised word embeddings from Reuters, neural image captioning that samples plausible sentences, autoencoder pre‑training, and over‑parameterized ImageNet models (tens of millions of weights) that reach near‑zero training error then generalize.
AI-written summary · about
chapters (33)
- 0:00Intro
- 0:53Mining for Structure
- 1:36Impact of Deep Learning
- 2:22Deep Generative Model
- 3:36Example: Understanding Images
- 5:09Caption Generation
- 5:29Talk Roadmap
- 6:26Learning Feature Representations
- 7:02Traditional Approaches
- 7:19Computer Vision Features
- 7:40Audio Features
- 8:24Neural Networks Online Course
- 8:51Feedforward Neural Networks
- 9:19Artificial Neuron
- 10:11Activation Function
- 12:15Single Hidden Layer Neural Net
- 12:37Multilayer Neural Net
- 13:02Capacity of Neural Nets
- 13:37Universal Approximation
- 15:50Stochastic Gradient Descend
- 17:27Computational Flow Graph
- 20:04Model Selection
- 21:19Early Stopping
- 23:42Mini-batch, Momentum
- 26:51Local vs. Distributed Representations
- 29:00Inspiration from Visual Cortex
- 31:56Why Training is Hard
- 34:39Unsupervised Pre-training
- 37:29Fine-tuning
- 46:35Dropout at Test Time
- 51:41Batch Normalization
- 57:16Optimization Tricks
- 57:57Visualization
