Orthogonal Statistical Learning

on YouTube ↗45m

" summary

Provides finite-sample guarantees that make nuisance estimation a second-order error source by using a two-stage sample-splitting meta-algorithm plus Neyman orthogonality, giving oracle rates under metric-entropy conditions and concrete bounds for variance-penalized ERM, neural nets and sparse high‑dim linear models. Proposes a robust GMM SEVER‑style filter that runs in polynomial time and achieves O(sqrt(epsilon)) recovery under an epsilon‑contamination model (demonstrated on IV and semi‑synthetic 401K/labor data), and a practical pipeline to estimate the Riesz representer via min‑max or squared‑loss for automated debiasing with neural nets and forests.

AI-written summary · about

" chapters (24)

1. 0:00Intro
2. 0:45The Ease of Use of Machine Learning
3. 1:20The Ease of (mis)Use of Machine Learning
4. 3:07Making CausalML Accessible to Every Decision-Maker
5. 10:24Automated Outlier Removal
6. 11:42Desiderata for a good robust estimator
7. 12:41Estimation via Moment Conditions
8. 14:15Generalized Method of Moments
9. 14:44Our Results
10. 17:17Two-part theorem
11. 17:43Algorithm and Key Ingredients
12. 21:29Example Application Theorem: IV Regression
13. 22:57Experiments
14. 24:37Application: NLSYM
15. 27:03Method of Moments with Nuisance Functions
16. 29:06De-biased moment
17. 34:13The Adversarial Approach
18. 36:40The Direct Loss Approach
19. 38:04Formal Theorem
20. 39:27Neural Network and Forest Heuristic Improvements
21. 40:41Multi-Tasking
22. 41:22Application: 401k
23. 41:57Application: Gasoline Demand
24. 43:56Beyond Linear Moments

" related talks

• 44mOrthogonal Machine Learning: Power and LimitationsLester Mackey · Robust and High-Dimensional Statistics (Simons Institute workshop, 2018)
• 31mRealizing RobustnessGautam Kamath · Robust and High-Dimensional Statistics (Simons Institute workshop, 2018)
• 01h 01mRobustness, Stochastics, Uncertainty 1Sasha Rakhlin, Ben Recht, Laurent El Ghaoui · Bridging Continuous and Discrete Optimization Boot Camp 2017
• 01h 02mSampling for Linear Algebra, Statistics, and Optimization IIMichael Mahoney · Foundations of Data Science Boot Camp 2018
• 01h 03mOverview of Statistical Learning Theory Part 2Nati Srebro · Modern Paradigms in Generalization Boot Camp 2024
• 48mComputational-Statistical Tradeoffs in Robust High-Dimensional EstimationIlias Diakonikolas · Robust and High-Dimensional Statistics (Simons Institute workshop, 2018)
• 37mEfficient Algorithms for Reliable Machine LearningAdam R. Klivans · The Role of TCS in Modern Machine Learning
• 44mActive Regression via Linear-Sample SparsificationEric Price · Real-Time Decision Making 2018