Model Validation Genesis
summary
Model validation in a legacy bank runs hundreds of models live and expands beyond checklists to changing regulations and the applications themselves; validation enforces standards, gives veto power for data teams to stop infeasible projects early, and aims to cut wasted prototypes while preserving short iterative delivery. The group prototypes with generative tools (he shows an image made with OpenAI DALL·E), leverages regulator backing, and reports partial success: management buy-in and fewer wasted projects but persistent legacy data and process debt remain.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:05Model validation role
- 3:17Hype versus reality
- 7:12Data as legacy
- 9:40Model risk mandate
- 11:32Model deployment mistakes
- 12:38Buzzword project dangers
- 14:02Vision versus legacy
- 16:28Models as metaphors
- 19:20Team empowerment
- 22:13Avoiding long projects
- 24:00Validation cost function
- 26:48Incentivizing improvement
- 28:14Models in business
- 29:24Q&A
related talks
30mHow to Properly Test ML Models & Data
26mTesting a Data Science Model
30mWhen Good Models Go Bad: The damage caused by wayward models and how to prevent it
24mModel Versioning: Why, When, and How
36mHow to review a model
19mApples, Oranges, and ML Models: Model Validation vs Benchmarking
31mEngineers guide for shepherding models in to production
42mTesting a Data Science Model