Portable and Fast - How to implement a parallel test runner
summary
Implements a multiprocess test runner using spawn over fork for Windows portability, falling back to TCP sockets on Windows and pipes elsewhere; uses Marshal.dump with custom marshal_dump/marshal_load and Base64-named constants to serialize test cases, and IO.select to multiplex real-time results. Runs workers in a pull-style producer/consumer model with bulk handoff at the test-case level to reduce setup/teardown overhead. Achieves 8s parallel vs 27s sequential on a 4-core demo, Thread and multiprocess runners released, Ractor runner in progress.
AI-written summary · about
chapters (8)
- 0:00Introduction and motivation
- 4:51Challenges of parallel execution
- 12:37Solutions for portability and speed
- 23:27Backward compatibility design
- 24:51Load balancing strategies
- 28:41Demo of parallel test runner
- 29:47Future work with Ractor
- 30:29Acknowledgments and closing
related talks
27mParallel testing with Ractors - Putting CPU's to work
58mWays to Parallelize Tests in JavaScript
40mInfinite improbability testing – execute all tests in parallel
32mParallel Testing World
26mConcurrency in Ruby: From fork() to Fiber
43mInfinite Improbability Testing- Execute All Tests in Parallel
59mWays to Parallelize Tests in JavaScript
28mRactors in Ruby 4.0: Message Passing Without the Pain