Etna: An Evaluation Platform for Property-Based Testing

on YouTube ↗28m

" summary

Etna is a multilanguage evaluation platform that runs property‑based testing experiments across Coq and Haskell, uses handcrafted mutants and mutation testing to measure bug‑finding, and ships workloads (data structures, languages, IFC) plus baselines. Experiments show concrete numbers: e.g. a workload with 36 tasks where generators find 14 bugs under 0.1s, six under 1s, four timed out at 60s; QuickCheck type‑based generators drop on red‑black trees versus bespoke and LeanCheck outperforms SmallCheck; fixes to FastCheck yield dramatic improvements.

AI-written summary · about

" chapters (12)

1. 0:00Property-based testing intro
2. 1:27Evaluating testing challenges
3. 3:54Introducing Etna platform
4. 5:41Etna evaluation strategies
5. 8:13Workloads and mutation testing
6. 11:23Etna use cases
7. 13:05Haskell framework comparison
8. 15:13QuickCheck generation strategies
9. 18:12Size effects in bug finding
10. 20:25Improving FastCheck with Etna
11. 22:58Etna conclusion
12. 23:41Q&A

" related talks

• 42mSurprise yourself with Property Based TestingMartin Hellspong · Øredev 2023
• 23mProperty based testing is a mindsetAndrea Leopardi · WebCamp Zagreb 2018
• 30mProperty-based Testing for Godly TestsGabe Scholz · ReactiveConf 2017
• 51mProperty-Based TestingLucy Mair · NDC London 2023
• 33mLets Generate TestsChris Keathley · ElixirDaze 2017
• 45mProperty based tests for the massesRomeu Moura · Code Mesh LDN 2018
• 47mProperty-Based TestingLucy Mair · NDC Oslo 2023
• 30mProperty Based Testing: From Theory to PracticeNicolas Dubien · Devoxx France 2021