Paradoxes in model training and evaluation under constraints
summary
Concrete claims: training on sales equals underforecasting, truncating to non‑stockout days induces hindsight bias, and censoring shifts a true mean demand of 9.7 to observed sales ≈8.6 and to ≈7.5 after truncation. Uses upper‑censored (Poisson) distributions, synthetic experiments and expectation calculations, and recommends computing the predicted capacity hit probability and bucket‑calibration for evaluation. Uses statsmodels for truncated fits, shows censoring removes information, and demonstrates that the probabilistic methods proposed actually align predicted and observed capacity hits.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 2:53Constraints break forecasting
- 7:12Capacity shapes demand
- 12:38Training under constraints
- 17:33Evaluating capacity hits
- 20:26Key takeaways
- 22:34Q&A
related talks
30mClimbing the causal ladder for fun and profit
33mUntangling the complexity of demand forecasting models: building a Market Simulator
31mFrom stockouts to happy customers: Proven solutions for time series forecasting in retail
26mParadoxes in Data Science
20mEvaluating Time-to-Event Models is Hard
17mCodifying Data Science Intuition
54mTime Series forecasting with NumPyro
30mKnowing what you don’t know matters- Uncertainty-aware model rating