Florian Rappl: Testing CLI Utilities
summary
Shows a homegrown test harness for the Pyro CLI that runs nearly 200 real end-to-end cases in about 15 minutes (previously over an hour) by using template directories to copy common scaffolding, a port allocator and resource manager to control network services, and cross‑platform matrix runs on Node.js with Playwright, TypeScript and Azure Pipelines. Tests live with the codebase (not a published framework), artifacts preserve CI temp output for debugging, and generated apps are validated in a browser.
AI-written summary · about
chapters (18)
- 0:00Testing CLI utilities
- 1:52CLI testing challenges
- 3:39Pyro CLI demo
- 5:25CLI testing framework requirements
- 8:56Test performance with templates
- 10:43Network resources in tests
- 13:09Testing tools and tech
- 15:14CI/CD pipeline for tests
- 17:24Test framework design
- 22:03Test suite and lifecycle
- 23:31CI/CD matrix testing
- 25:59Azure Pipelines matrix
- 26:41Test results and reports
- 28:31Single test case demo
- 30:59Debugging failed tests
- 32:06Built our own test framework
- 33:35Summary
- 33:55Q&A
related talks
51mAdvanced Playwright Techniques for Flawless Testing
29mPlaying your tests wright
11mHow to make cheap e2e tests
48mSupercharged Testing: AI-Powered Workflows with Playwright + MCP
36mCLI applications & TDD: Never write scripts again
18mAt the top of the pyramid: Playwright testing at scale
21mEveryone Can Easily Write Tests
25mPlaywright Test Runner