The Importance of Modeling Data Collection
summary
Argues that systematic observation bias skews ML estimates and routine validation hides it, shows two concrete fixes: model the observation probability as p(obs|Y) (collapsing high-dimensional X dependence) and adapt nonparametric learners such as Gaussian Process regression with an extra observation-loss term. Demonstrations use white blood cell measurements and software project profit examples, and a key claim is that a small random subset of data lets the model identify the selection function and recover accurate predictions in low-data regions.
AI-written summary · about
chapters (11)
- 0:00Intro
- 1:26Data Collection
- 2:44Low Profit
- 3:56Selection Bias
- 4:45Machine Learning Model
- 5:11Model Data
- 7:50What if we dont know
- 8:45Example
- 14:42Conclusions
- 15:12Questions
- 18:09Contact Info
related talks
31mLife cycle of data becoming a model
39mBeyond Bias: Crafting Fairness in Pythonic Machine Learning Systems
25mWhy Good Models Fail After Deployment
33mHidden Biases In Data: Moving Beyond Black Box Models
30mFrom Black Box to a Known Quantity: How to Build Predictable, Reliable ML-based
39mDoing Data: The Critical Process of Data Preparation (...)
30mWhen Good Models Go Bad: The damage caused by wayward models and how to prevent it
30mMachine Learning Techniques for Class Imbalances & Adversaries