Codifying Data Science Intuition
summary
Decision-theory minimax decision function selects the model that minimizes worst-case error across consecutive, temporally independent holdouts; implemented with a fixed pool of ~40 models per series so computation stays bounded. Experiment uses 856 retail daily series with exogenous indicators, producing ~100,000 error measurements and using RMSPE selection. Result: minimax is slightly worse than choosing the single-model that minimizes RMSPE on one test split, outperforms a naive weekly seasonal baseline, and ensembles beat both.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:51Time series model selection
- 4:03Time series train/validate/test
- 7:26Minimax model selection
- 9:25Experiment setup
- 11:33Error metrics and visuals
- 14:33Results and performance
- 15:58When to use minimax
- 16:19Closing
related talks
09mTowards Credible and Effective Data-Driven Decision-Making
29mEfficient Minimax Optimal Strategies for Universal Prediction
32mThe Dangers of Mindless Forecasting
55mLearning Minimax Estimators Via Online Learning
40mDecision-Making Under Miscalibration
12mCausal Inference And Data-Driven Decision-Making
01h 11mA Theory of Multi-objective Machine Learning
35mWhat Failure Taught Me About Building High-Stakes Models