Auto-testing for situational awareness

on YouTube ↗55m

" summary

Auto-tests focus on timelines not just pass/fail: nightly GitLab CI runs on Alpha correlate test durations, executed-test counts and reruns to reveal that a Wi‑Fi-default-off change blocks the out‑of‑box flow, causes stuck GUI navigation and breaks page object model tests. Strategy uses YAML retries, per-job cleanup scripts and a page object model to recover tests, ELK/ Elastic stack dashboards to spot trends, and a small open‑source Python behave GUI to review Gherkin steps — together they catch regressions early and distinguish genuine failures from flakiness.

AI-written summary · about

" chapters (16)

1. 0:00Introduction
2. 1:35Invest in automated testing
3. 3:40Situational awareness in testing
4. 7:12Observability and testing pyramid
5. 9:46Correlating timelines for insights
6. 11:56Cost of unreliable test runs
7. 13:22Detecting regressions case study
8. 19:56Situational awareness in action
9. 22:26Out-of-box regression
10. 25:17Broken tests after changes
11. 28:50Strategies for more tests
12. 34:01Page object model
13. 40:49Avoiding GUI dependency pitfalls
14. 43:42Demonstrating Behave GUI tool
15. 50:32Focusing on trends
16. 51:39Tester-developer collaboration

" related talks

• 53mTest architecture is a real thingJames Westfall · NDC TechTown 2024
• 21mThe Essential Pillars of Automation Testing for SuccessSagar Jitendra Mahendrakar · DevConf.CZ 2025
• 57mAdvanced Testing Techniques: Tips from the trenchesRob Moore, Graeme Foster · NDC Sydney 2018
• 56mTestOps at scaleJames Westfall · NDC TechTown 2021
• 27mQuest for Adequate Autotest CoverageAlexander Pushkarev · DelEx Conf 2018
• 50mAsaf Shochet Avida, Eitan Noy: Why My Amazing Test Automation Project CrumbledAsaf Shochet Avida, Eitan Noy · TestCon Europe 2024
• 55mThe art of living with broken thingsJames Westfall · NDC TechTown 2023
• 32mLiving on Crotches or UI Automation AntipatternsVadim Zubovich · TestCon Moscow 2017