Testing with two failure seeking missiles: fuzzing and property based testing

on YouTube ↗38m

" summary

Covers concrete results: Hypothesis generates thousands of guided random tests, finds counterexamples and then shrinks them to minimal failing inputs; American fuzzy lop instruments C code, uses genetic mutation and reports unique crashes (ffmpeg yielded ~1,000 bugs); Python-AFL hooks AFL to the Python runtime so C-level crashes surface from Python inputs. Demos show hundreds of thousands to millions of runs and produce reproducible crash files and reduced examples to add to a deterministic test suite.

AI-written summary · about

" chapters (5)

1. 0:00Start
2. 2:25Artist's impression of edge cases
3. 3:03So how to create test data?
4. 14:33American fuzzy lop (afl)
5. 17:02Types of bugs found

" related talks

• 52mStructure aware fuzzingRéka Kovács · Meeting C++ 2018
• 43mOmg, how do I write software that isn't a ticking timebomb!?Eivind Jahren · NDC TechTown 2024
• 01h 07mSufficiently Advanced TestingZac Hatfield-Dodds · PyCon AU 2019
• 01h 31mIntroduction to Property-Based TestingZac Hatfield-Dodds, Ryan Soklaski · PyCon US 2023
• 53mFuzz Testing for Fun and ProfitMELISSA BENUA · BSides Huntsville 2021
• 51mProperty-Based TestingLucy Mair · NDC London 2023
• 42mSurprise yourself with Property Based TestingMartin Hellspong · Øredev 2023
• 53mFuzzing 101Kamil Frankowicz · 4Developers 2022 Online