Distributional Robustness, Learning, and Empirical Likelihood

on YouTube ↗34m

" summary

Claims that modest distributional robustness recovers a variance penalty and gives usable certificates, implemented two concrete tools: a convex reweighting using Empirical Likelihood Upper Confidence Bound with chi-squared divergence that solves in near-ERM time and raises Reuters recall on the rare “economics” class from ~68% to ~80% (~10% gain), and a Wasserstein balls approach that, with smooth nonlinearity approximations, yields neural-network training resistant to common white‑box attacks (FGSM, PGD) and produces conservative robustness certificates, but requires smoothness and large λ to avoid NP‑hardness.

AI-written summary · about

" chapters (21)

1. 0:00Intro
2. 0:25Motivation
3. 0:35Challenge one: Curly fries
4. 1:26Challenge two changes in environment
5. 1:47Challenge three adversaries
6. 2:58Stochastic optimization problems
7. 3:47Distributional robustness
8. 5:08Vignette one regularization by variance
9. 6:44Optimizing for bias and variance
10. 7:21Empirical likelihood and robustness
11. 10:07Optimal bias variance tradeoff
12. 10:54Experiment: Reuters Corpus (multi-label)
13. 14:56Vignette two: Wasserstein robustness
14. 15:47Challenges
15. 19:56Duality and robustness
16. 20:48Stochastic gradient algorithm
17. 21:55A certificate of robustness
18. 22:46Digging into neural networks
19. 23:57Experimental results adversarial classification
20. 27:39Reading tea leaves
21. 28:26Reinforcement learning?

" related talks

• 12mDistribution-Free RobustnessSitan Chen · Meet the Fellows Welcome Event
• 53mDesigning Robust LearnersJacob Steinhardt · Emerging Challenges in Deep Learning (Simons Institute workshop, 2019)
• 01h 17mRobust Optimization and GeneralizationJohn Duchi · Modern Paradigms in Generalization Boot Camp 2024
• 22mPanel DiscussionChristopher Harshaw, Liam Solus, Arnab Bhattacharyya · Learning from Interventions 2022
• 01h 28mMulti-Distribution Learning, for Robustness, Fairness, and CollaborationNika Haghtalab · Data-Driven Decision Processes Boot Camp 2022
• 01h 24mAdversarial ExamplesSebastien Bubeck · Deep Learning Boot Camp 2019
• 01h 01mRobustness, Stochastics, Uncertainty 1Sasha Rakhlin, Ben Recht, Laurent El Ghaoui · Bridging Continuous and Discrete Optimization Boot Camp 2017
• 42mBeyond Adversarial Approaches on Distributional RobustnessQi Lei · Adversarial Approaches in Machine Learning 2022