Tutorial on Deep Learning II
summary
Claims concrete wins: AlexNet halved error on ImageNet (1.2M images), 8 layers, ~50M parameters, ~18% top‑5 and removing one FC layer drops only ~1% while saving 16M params; Inception redesign cuts to ~5M params and reaches ~6.7%; ResNet uses skip connections to scale to 151 layers. Models are used as plug‑in features (often hardwired) and transfer surprisingly well to other domains (even medical images). Training tools include convolutional nets, contrastive divergence for RBMs, layerwise pretraining improving a variational bound, and autoregressive Pixel models that actually generate realistic images.
AI-written summary · about
chapters (39)
- 0:00Intro
- 0:20Computer Vision
- 1:40ConvNets: Examples
- 2:09ImageNet Dataset
- 2:32Important Breakthrough • Deep Convolutional Nets for Vision (Supervised)
- 3:59How to Choose Architecture
- 4:36AlexNet
- 5:37GoogleNet
- 9:55Residual Networks
- 11:15Choosing the Architecture
- 13:55Talk Roadmap
- 14:54Unsupervised Learning
- 17:49Sparse Coding: Training
- 18:25Sparse Coding: Testing Time
- 19:24Interpreting Sparse Coding
- 21:56Loss Function
- 22:29Denoising Autoencoder
- 23:34Learned Filters
- 24:01Predictive Sparse Decomposition
- 24:56Stacked Autoencoders
- 25:42Deep Autoencoders
- 26:52Information Retrieval
- 27:19Semantic Hashing
- 27:52Searching Large Image Database using Binary Codes Map images into binary codes for fast retri
- 28:30Deep Generative Model
- 30:27Fully Observed Models
- 32:51Restricted Boltzmann Machines
- 34:10Learning Features
- 35:13Model Learning
- 37:11Approximate Learning
- 38:06Approximate ML Learning for RBMs
- 39:17RBMs for Real-valued Data
- 39:57RBMs for Word Counts
- 41:47Collaborative Filtering
- 43:05Product of Experts
- 45:15Deep Belief Network
- 48:57DBN Layer-wise Training
- 50:49Why this Pre-training Works?
- 53:22Learning Part-based Representation
