Test Automation to the Max

on YouTube ↗36m

" summary

Yandex describes a pipeline that keeps human-readable test scenarios alongside code, links them to automated tests and executes over 17,000 desktop functional scenarios with the Hermione test runner plus locked-data screenshot checks. Manual regression is outsourced to about 9,000 vetted crowd workers via Yandex Toloka using honey‑pot tasks, with a custom TestPal tool exporting steps to Toloka and reimporting failures as tickets. The system runs daily but needs heavy maintenance (about 50–70% of dev effort) and ongoing work to handle flaky screenshot diffs.

AI-written summary · about

" chapters (4)

1. 0:00Start
2. 7:02Why Manual Testing?
3. 18:06Search Interfaces
4. 19:42Documentation Relevance Check

" related talks

• 46mHow to Make Automated Tests Useful: The Evolution of Tests in Yandex IDDmitriy Andriyanov · HolyJS 2024 Autumn
• 32mSpeeding Up Testing Using the CrowdDmitriy Savin · TestDriven Conf 2022
• 45mHow Not to Fear Breaking Production? QA Automation Lessons with TestplaneNikolay Markov · HolyJS 2025 Spring
• 56mTestOps at scaleJames Westfall · NDC TechTown 2021
• 28mBuilding an automated testing suite...Charles Sonigo, Jean-Baptiste Louazel · Demuxed 2019
• 32m“The Yahoo! Story” How Yahoo! Embraced Automated Testing to...Neil Manvar · DevOps Enterprise Summit 2015
• 53mTransforming testing methodologies: From manual to robust strategiesTodd Eaton · Breakpoint 2020
• 41mAlexander Gyulai: A Practical Guide to Transforming Quality Assurance with Intelligent SolutionsAlexander Gyulai · TestCon Europe 2024