Testing a Data Science Model
summary
Claims a practical workflow: models use genetic algorithms producing stochasticness, testing relies on a baseline run compared to new runs with an accepted 3-5% threshold, and teams ship every two-week sprint. Built tooling includes a UI to trigger models, SQL and Excel for validation, and front-end dev tools for endpoint checks. Tests verify data pipelines (no dropped or duplicated rows), match anonymized file sizes to production, and only automate pass/fail checks when thresholds suffice.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:32Data science and model testing
- 3:58Data science models overview
- 8:29Model testing process
- 10:34Key testing aspects
- 12:01Feature and regression testing
- 15:53Practical model validation
- 17:16Testing complexities
- 19:02Model testing tips
- 21:33Summary
- 22:13Q&A: Unit tests
- 24:21Q&A: Tester vs data scientist
related talks
42mTesting a Data Science Model
30mHow to Properly Test ML Models & Data
42mDo Your Homework! Writing tests for Data Science and Stochastic Code
31mTesting Machine Learning Models
31mLife cycle of data becoming a model
59mDevOps for Data Science
45mChris Harbert: Testing AI at the Scale of Big Data
25mWhy Good Models Fail After Deployment