Automated Tests Are Not Enough
summary
Collected notes show 21 real bugs over three years that automated tests missed, so manual exploratory testing is added to unit and integration suites. Examples include changing currency that unexpectedly clears the call type, GUI-submitted numbers arriving as strings not floats, a pre-calculated sum drifting out of sync with trade items, API calls failing when the frontend sends full payloads versus the API’s partial updates, and misreading Refinitive’s date rollout. Uses local/test/prod runs, logs, feature flags and parallel implementations to detect and contain issues.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:37Structure
- 2:53Automatic Tests
- 9:31Explore It
- 9:59Checking vs Exploring
- 14:18Why Manual Testing
- 16:35Examples
- 19:58First example
- 22:56Second example
- 24:09Third example
- 28:02Fifth example
- 31:53Lessons
- 34:15Testing in Production
- 37:39Recap
related talks
42mOpening Keynote: Blunders in Test Automation
31mExploratory Testing on Computer Interfaces (APIs)
14m3 Automation Testing Mistakes & How to Correct Them?
26mAutomated testing what is the point
01h 48mNo Such Thing as Manual Testing and Other Confusions
24mWHY ARE WE FAILING WITH TEST AUTOMATION?
35m82 Bugs I Collected in a Year You Won’t Believe Made It to Production
16mIsn’t Test Automation a Silver Bullet?