Variable Selection with Error Control: Another Look at Stability Selection

on YouTube ↗48m

" summary

Introduces complementary-pair stability selection: apply any base selector to random half-samples and their complements, aggregate selections, and keep variables chosen at least proportion t (t>0.5). Proves a finite-sample bound on the expected number of falsely selected (low-selection-probability) variables that scales like Q·(something)/((2t−1)P), so with P=1000, Q=50 and t=0.6 the bound cuts low-probability picks to about one quarter of the base method. Improves the bound by assuming unimodality or r = -1/2 concavity; simulations (n=200, p=1000, lasso base) show better false-positive control but higher false negatives if tuned aggressively.

AI-written summary · about

" chapters (20)

1. 0:00VARIABLE SELECTION WITH ERROR CONTROL: ANOTHER LOOK AT STABILITY SELECTION
2. 0:54What is Stability Selection?
3. 4:22A general model for variable selection
4. 6:15How does Stability Selection work?
5. 15:09Error control discussion
6. 15:45Complementary Pairs Stability Selection
7. 17:07Worst case error control bounds
8. 22:59Illustration and discussion
9. 25:34Proof
10. 30:21Bounds with no assumptions whatsoever
11. 32:10Can we improve on Markov's inequality?
12. 32:49Improved bound under unimodality
13. 33:35Extremal distribution under unimodality
14. 35:14The r-concavity constraint
15. 36:57Extremal distribution under r-concavity
16. 37:08T= -1/2 is sensible
17. 37:53Reducing the threshold
18. 39:27Simulation study
19. 42:41Simulation results
20. 44:04Summary

" related talks

• 22mPanel DiscussionChristopher Harshaw, Liam Solus, Arnab Bhattacharyya · Learning from Interventions 2022
• 32mHypothesis selection with computational constraintsMaryam Aliakbarpour · Extroverted Sublinear Algorithms 2024
• 38mWhy giving your algorithm ALL THE FEATURES does not always workThomas Huijskens · PyData London 2018
• 41mGeneralization Bounds for Uniformly Stable AlgorithmsVitaly Feldman · Robust and High-Dimensional Statistics (Simons Institute workshop, 2018)
• 01h 01mRobustness, Stochastics, Uncertainty 1Sasha Rakhlin, Ben Recht, Laurent El Ghaoui · Bridging Continuous and Discrete Optimization Boot Camp 2017
• 54mOff-the-shelf Algorithmic StabilityRebecca Willett · Unknown Futures of Generalization 2024
• 01h 02mSampling for Linear Algebra, Statistics, and Optimization IIMichael Mahoney · Foundations of Data Science Boot Camp 2018
• 33mBalancing Covariates In Randomized Experiments: The Gram--Schmidt Walk DesignChristopher Harshaw · Learning from Interventions 2022