Testing the Hard Stuff and Staying Sane
summary
Randomized QuickCheck specs drive millions of generated call sequences against real C and Erlang (Adlang) code, then apply shrinking to produce tiny failing traces. Concrete wins: a circular‑buffer size bug fixed after a 3‑step repro, a CAN‑ID masking error uncovered in message priority handling, and race conditions in the debts file system that reduce months of debugging to five‑call reproductions. Industrial scale: 20,000 lines of specs test ~1,000,000 lines of C from six suppliers and found 200 bugs.
AI-written summary · about
chapters (19)
- 0:00Start
- 0:29Why is testing hard?
- 2:49The commonest usage
- 4:03Example: a circular buffer
- 5:01Size is a little tricky...
- 5:25State Machine Models
- 6:58Code Fragments: specifying get
- 17:54Lessons
- 21:28Practice
- 24:25A Bug in a vendor's CAN stack
- 29:27A Unit Test in Erlang
- 29:56Modelling the dispenser
- 30:50A Parallel Unit Test
- 32:24Another Parallel Test
- 33:11Deciding a Parallel Test
- 37:48QuickCheck Specification
- 40:16Bug #3
- 42:33Bug #4
- 47:07Property Based Testing
related talks
42mSurprise yourself with Property Based Testing
33mLets Generate Tests
15mIronSpec: Increasing the Reliability of Formal Specifications
45mTesting Web Services with QuickCheck
38mLiving with Spec
15mBennet: Randomized Specification Testing for Heap-Manipulating Programs
12mBuild bravely: Delivering risky projects
27mBreak It Yourself