Deep Learning Boot Camp 2019
as the organisers put it
The Boot Camp is intended to acquaint program participants with the key themes of the program. It will consist of four days of tutorial presentations from the following speakers: Sasha Rakhlin (University of Pennsylvania) Peter Bartlett (UC Berkeley) Jason Lee (University of Southern California) Nati Srebro (Toyota Technological Institute at Chicago) Kamalika Chaudhuri (UC San Diego) Matus Telgarsky (University of Illinois at Urbana-Champaign)
who it's for
theoretical machine learning researchers, deep learning researchers, machine learning PhD students, postdoctoral researchers, research scientists, statisticians interested in learning theory
topics
Rademacher complexity bounds; neural tangent kernel limits; overparameterized network dynamics; implicit regularization of gradient descent; max-margin bias for logistic loss; adversarial robustness and randomized smoothing; ReLU depth and approximation; nearest neighbors and k-NN rates; kernel methods and benign overfitting; matrix factorization and min-nuclear-norm
| title | author | length |
|---|---|---|
| Generalization I | Peter Bartlett, Sasha Rakhlin | 01h 17m |
Proves a concrete generalization claim: the perceptron update makes at most 1/γ^2 mistake bound on any sequence with margin γ, and a one‑pass run yields… | ||
| Approximation Power | Matus Telgarsky | 01h 14m |
Shows that continuous univariate functions are representable as integrals of step/ReLU atoms by sampling the derivative or second derivative, and that… | ||
| Generalization II | Peter Bartlett, Sasha Rakhlin | 01h 23m |
Shows that uniform deviation between population and empirical losses is essentially captured by Rademacher averages using symmetrization with a ghost sample… | ||
| Generalization III | Peter Bartlett, Sasha Rakhlin | 01h 18m |
Uses Rademacher symmetrization to reduce uniform deviations to analyzable averages and gives classification bounds like sup deviation = O(sqrt(d log n / n))… | ||
| Recent Developments in Over-parametrized Neural Networks, Part I | Jason Lee | 01h 14m |
Shows that widening a network often fixes optimization: a realizable two‑layer teacher with 50 neurons resists SGD when fitted with 50 units but reaches… | ||
| Generalization IV | Peter Bartlett, Sasha Rakhlin | 01h 22m |
Compares K‑nearest neighbors and kernel smoothing with explicit bias–variance decompositions, deriving the minimax rate N^{-2/(2+d)} under Lipschitz… | ||
| Adversarial Examples | Sebastien Bubeck | 01h 24m |
Adversarial robustness remains unsolved empirically and theoretically: on CIFAR‑10 with epsilon=8 the best reported robust accuracy is ~57% and on ImageNet a… | ||
| Recent Developments in Over-parametrized Neural Networks, Part II | Jason Lee | 01h 14m |
Survey of recent theory connecting wide nets to neural tangent kernel and random features frameworks, with concrete claims: kernel limit arises as width → ∞… | ||
| Nearest Neighbors I | Kamalika Chaudhuri | 01h 08m |
K‑nearest neighbors is universally consistent when K → ∞ and K/N → 0, and regression rates scale like N^{-2/(2+D)} with the optimal K ≈ N^{2/(2+D)}, exposing… | ||
| Implicit Regularization I | Nati Srebro | 01h 17m |
Overparameterized nets and matrix completion have many global minima, yet the chosen optimizer determines which beach the solution lands on: SGD vs Adam… | ||
| Implicit Regularization II | Nati Srebro | 01h 23m |
Claims that optimization selects specific solutions: for underdetermined least squares gradient descent from zero yields the minimum norm solution; for… | ||
| Generative Models | Elchanan Mossel | 01h 08m |
Models where labels are generated top-down on a tree get concrete statements: the optimal decoder is computed by belief propagation, phylogenetic algorithms… | ||
| Nearest Neighbors II | Kamalika Chaudhuri | 01h 05m |
One nearest neighbor has robustness radius → 0 at points where mu is continuous and eta(x)∈(0,1) so accuracy can remain high while robustness vanishes as n→∞… | ||
| nothing matches | ||