Testing a Data Science Model
summary
Walks through a concrete testing process: verify datasets and shapes, run Delta run regression comparisons, expect model stochasticness and set thresholds 0–5% for acceptable drift. Uses SQL Excel basic statistics DevTools and Mirror. Builds an anonymization automation for about 18–20 client files that cuts a half day task to ~5 minutes and reduces upload incidents. Emphasizes edge case testing and exploratory work plus unit tests from data code and retrain-if-above-threshold practice.
AI-written summary · about
chapters (20)
- 0:00Introduction to testing models
- 2:01What is a data science model
- 5:59Predictive analytics explained
- 9:13Data types in modeling
- 13:11How models work
- 16:03Testing black-box models
- 18:07Step-by-step model testing
- 23:28Practical testing example
- 24:30Complexities in data testing
- 26:59Tips for effective testing
- 28:24Tools for hybrid testers
- 29:27Summary
- 30:09Q&A: Test automation
- 32:16Q&A: Thresholds and variations
- 33:20Q&A: Genetic algorithms
- 34:47Q&A: Setting test thresholds
- 35:49Q&A: Poor database structures
- 36:31Q&A: Teams without data scientists
- 37:33Q&A: Bug types in testing
- 41:06Closing remarks
related talks
26mTesting a Data Science Model
39mDoing Data: The Critical Process of Data Preparation (...)
30mHow to Properly Test ML Models & Data
01h 30mAdvanced Software Testing for Data Scientists
01h 29mAdv. Software Testing for Data Scientists
30mDetecting drift: how to evaluate and explore data drift in machine learning systems
42mDo Your Homework! Writing tests for Data Science and Stochastic Code
50mData-Driven Engineering