Do Your Homework! Writing tests for Data Science and Stochastic Code
summary
Concrete method for testing nondeterministic code: write gold tests (human-labeled JSON examples), use Pytest examples with fixtures and xfail, and benchmark models over time. Implements random checks with scipy.stats normaltest (p‑value behavior noted) and an sklearn text-classification pipeline using SGD and grid search, recording a baseline score (example run gives p‑value 0.939 vs target 0.95). Generates HTML and coverage reports, compares plots via matplotlib, and keeps reproducible scripts to rerun and track regressions.
AI-written summary · about
chapters (15)
- 0:00Testing Data Science code
- 1:04Homework teaches testing
- 3:10Test-driven mindset
- 4:55Benefits for data scientists
- 6:01Supply chain testing need
- 9:37Setting up pytest
- 12:27Basic unit tests
- 18:42Edge cases and validation
- 21:18Testing random outputs
- 25:20Stochastic ML models
- 30:15Advanced pytest features
- 34:11HTML reports and coverage
- 36:24Conclusion and next steps
- 37:04Q&A: Mocking datasets
- 38:30Q&A: Convincing teams
related talks
01h 29mAdv. Software Testing for Data Scientists
26mTesting a Data Science Model
01h 30mAdvanced Software Testing for Data Scientists
42mTesting a Data Science Model
29mTDD shouldn't be TDDious
27mPyTesting the Limits of Machine Learning
59mTesting in Python
22mWriting tests when the code is already there: Golden Master technique