Adv. Software Testing for Data Scientists
summary
Workshop demonstrates concrete practices: use pytest for low-boilerplate unit tests, numpy/pandas-specific assertions like pandas.assert_frame_equal, property-based testing with hypothesis that generates ~100 examples and finds a double-slash slug bug (split('/', 1) fixes it), and test doubles/mocks with controlled mock side_effect to replace REST or S3 calls. Codebase is a Kickstarter dataset, a scikit-learn logistic regression pipeline serialized to joblib, and the demos show which tests fail, which inputs are generated, and the exact fixes that make them pass.
AI-written summary · about
chapters (13)
- 0:00Welcome
- 3:37Setup tutorial environment
- 8:46Testing matters for data scientists
- 17:35Python testing tools overview
- 23:15Unit testing with pytest
- 26:28Pandas test patterns
- 32:29Parameterized pytest testing
- 43:03Maintainable test writing
- 54:20Property-based testing intro
- 1:01:31Hypothesis input generation
- 1:18:07Static type checking
- 1:19:34Mocking dependencies
- 1:28:09Q&A
related talks
01h 30mAdvanced Software Testing for Data Scientists
39mUnit Testing for Data Scientists
01h 28mTesting with Pytest for Data Science
42mDo Your Homework! Writing tests for Data Science and Stochastic Code
01h 11mTest-Driven Data Analysis
28mHow to not pull your hair out while providing data to the business
39mTesting Big Data Applications (Spark, Dask, and Ray)
25mTesting Data Pipelines