Rules of Engagement QUESTIONS TO ASK BEFORE YOU START AUTOMATING
summary
Checklist of concrete questions and hard lessons: projects fail because there is no test data strategy (three months of POC but no usable tests), because teams enact automation for automation's sake (2–3k brittle tests with ~70% failing), or because process and contracts prevent early testing (stable ebook deploys blocked by late tests). Ask what goal is being measured (use SMART), who writes and maintains tests, acceptable test runtime, tech stack and infra constraints, where data comes from, unit test coverage, and whether code or process changes are cheaper than fragile UI automation.
AI-written summary · about
chapters (29)
- 0:00Intro
- 0:24There are no answers here
- 6:56System is old and not documented. There is no one with enough knowledge to even design mocked data
- 8:52Example II System for tracking shipments
- 10:13Test where poorly written and not maintained. Simple change took more a day.
- 12:02Because company understanding of Agile required to have automation (This is not a joke)
- 14:22The process was not prepared for new delivery goal.
- 16:16A lot of time automation problems starts in process or design. Rarer in technical aspect
- 17:22People are not buying software
- 17:57Team is not "buying" automation
- 18:26Automation should be treated same as production code.
- 20:03Testability is part of architecture
- 21:33Automation (and testing) Is part of project • Testability need to be built into project • Testing need to bring value
- 22:02Never start with choosing tools!
- 22:20What is current delivery process? What is current testing process?
- 25:54Does the goal brings value to team and stakeholders?
- 27:24Who will write and maintain the test?
- 27:39What happens when test fails?
- 28:02What are relations between dev and testers? Are there any testers?
- 28:42What can be changed in code to improve testability?
- 29:58How are we measuring progress?
- 30:13Can we make changes in delivery process? How much can we change?
- 33:10What is our tech stack?
- 35:12How often are we running the test?
- 35:35What is the maximum acceptable length for test run?
- 37:04Who is responsible for writing and maintaining documentation?
- 37:48What is the process for creating tests?
- 38:52Testability is part of application architecture.
- 39:313 Most important Questions 1. What are current bottlenecks? 2. What is the goal of automation? 3. What is the state of unit tests?
related talks
24mWHY ARE WE FAILING WITH TEST AUTOMATION?
38mDo We Really Need A Test Automation Engineer?
21mThe Essential Pillars of Automation Testing for Success
08mReviewing your test automation strategy"
40mThe Test Automation Firehose
54mTest Automation Transformation Avoiding the 7 Common Pitfalls
18mEffective Test Automation Framework Design
46mThe Practical Automation Playbook