Infinite Improbability Testing- Execute All Tests in Parallel
summary
Claim: 100,000 tests in ~20 minutes. Implementation: an open‑source distributed test runner plus agents on remote VMs; large run uses 10 machines × 16 threads = 160 simultaneous with each test ≈1s. Measured wins: 4,500 UI tests drop from ~16 hours sequential to ~4 hours on 4 VMs and the big synthetic run completes in under half an hour, often >10× faster than the native runner. Features used include balancing by prior runtimes, retries of failures, result merging and cross‑platform execution; the system works at that scale.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 2:21The state of test execution
- 5:04Parallel testing strategies
- 7:48Introducing Mesa
- 10:59Using and configuring Mesa
- 13:28Demo and performance results
- 17:35Deep dive into setup
- 21:06Comparison with alternatives
- 24:07Architecture and design
- 26:45Runner coordination logic
- 30:26Getting started and scaling
- 32:51Q&A and troubleshooting
related talks
40mInfinite improbability testing – execute all tests in parallel
33mFrom Hours to Minutes: Run Thousands of CI Tests in Just Minute
09mTest Time Execution! Why it Can't be Ignored?
26mThe Power of Atomic Automated Tests: Our experience of Improved Test Efficiency...
32mParallel Testing World
38mThe Power of Atomic Automated Tests
31mTests Are What You Want, What You Really Really Want
34m1.5 Billion Tests Per Day