Testing stochastic AI models with Hypothesis
summary
Uses the Hypothesis library to generate and shrink failing examples, stores failing cases in a local database for repeatable random tests, and combines property-based testing with metamorphic relations to test medical-image feature extraction. Concrete checks include asserting features persist when adding extra connected components (generated via array strategies, composite draws and binary dilation) and asserting rotation invariance while mocking location-dependent calculations. Evidence: reproducible failures, automatic shrinking, numpy/pandas strategy support.
AI-written summary · about
chapters (23)
- 0:00Intro
- 0:27About me
- 2:03Table of Content
- 3:16Example-based testing- example
- 3:44Example-based testing - issues
- 4:40Example-based testing - merge_sort
- 6:40Property: Commutativity
- 6:58Property: Invariant functions
- 7:15Property: The test oracle
- 7:55Property-based testing
- 8:34What are the properties in the example?
- 9:35Metamorphic Testing
- 10:59Metamorphic Relations
- 13:05Hypothesis Library
- 14:31Hypothesis basic strategies
- 15:53merge_sort test
- 16:11Define you own strategy
- 17:02Transforming data functions
- 17:33Debug hypothesis strategies
- 18:29Repeatable random testing
- 19:03Shrinking
- 21:10Additional Components
- 22:20Rotate the image
related talks
50mProperty based testing with Hypothesis
41mProperty Based Testing in Python with Hypothesis
27mTesting Beyond Examples: Property Based Testing with Hypothesis
41mProperty Based Testing with Hypothesis: Stronger Tests, Less Work
32mMore testing with fewer tests
31mAdventures in not writing tests
27mAdventures in not writing tests
01h 07mSufficiently Advanced Testing