Designing Robust Learners
summary
Shows that projecting the corrupted distribution to the closest valid model and outputting that model’s parameter is information-theoretically optimal up to constants, with the error controlled by a modulus of continuity; proves thin-tailed families satisfy a resilience condition so Gaussians drop error from the naive ε√d down to O(ε). Finite-sample work uses sup over linear tests (VC bounds) to get d/N rates, and Wasserstein extensions use couplings and a friendly perturbations fixed‑point to generalize but leave computationally efficient algorithms and a troubling N^(-1/3) bug as open.
AI-written summary · about
chapters (14)
- 0:00Intro
- 0:34Motivation
- 1:26The Difficulty
- 3:35Context and Overview
- 5:08Setting
- 12:08Warm-up: TV, mean estimation
- 15:34Modulus: Examples
- 22:30Finite-sample estimation
- 32:06General TV case
- 33:23Resilience: Arbitrary loss
- 35:01Example: Linear regression
- 40:55Example: Covariance estimation
- 45:04Friendly perturbation: formal definition
- 47:17Resilience for W
related talks
34mDistributional Robustness, Learning, and Empirical Likelihood
37mEfficient Algorithms for Reliable Machine Learning
48mComputational-Statistical Tradeoffs in Robust High-Dimensional Estimation
01h 00mAlgorithmic High Dimensional Robust Statistics I
36mPrivately Learning High-Dimensional Distributions
01h 04mAlgorithmic High Dimensional Robust Statistics II
31mRealizing Robustness
41mEfficient Algorithms for Outlier-Robust Regression