UI Testing at Pinterest

on YouTube ↗45m

" summary

Pinterest runs a scaled UI test system that shrinks a six‑hour suite into commit‑time checks by moving to Firebase Test Lab with Flank sharding, using emulators (Nexus lowres), Espresso and Kotlin, and cutting runtime cost about 30% cost reduction. The infra handles ~17,000 builds and ~2.2M UI runs per month for ~30 developers, defines flakiness as inconsistent results, retries tests up to 2 retries (3 runs), enforces test ownership, provides Metro dashboards, Pin Tester and a login manager, and requires stabilization (e.g. 100‑run validation) before a test blocks commits.

AI-written summary · about

" chapters (11)

1. 0:00Introduction to UI testing
2. 1:17History of UI testing at Pinterest
3. 4:45Speeding up Android UI tests
4. 6:32Defining and measuring test stability
5. 9:40Core principles for UI testing
6. 15:41Android technologies and tools
7. 17:27Cross-platform testing tools at Pinterest
8. 19:59Introducing Metro testing dashboard
9. 27:49Metro features and dashboards
10. 31:43Recommendations and next steps
11. 37:28Q&A and closing remarks

" related talks

• 44mRock-Solid UI and Instrumentation TestingSinan Kozak · droidcon Berlin 2021
• 23mUI Testing for Fun and ProfitSarah Olson · try! Swift Tokyo 2018
• 21mTasting Tests at CookpadKazuaki Matsuo · try! Swift Tokyo 2017
• 34mSetting up Android UI Testing on CIDmytro Shuba · droidcon Berlin 2023
• 45mTesting the web without testers — success or failureTatyana Ryzhenkova, Sergey Zvyagin · FrontTalks 2016
• 29mHow to Shorten UI Test Run Timestarappo · DroidKaigi 2018
• 22mTwo years of UI TestingCamin, Bigagnoli · Swift Heroes 2018
• 26mEfficient Android Builds at Scale: Lessons from PinterestDavid Chang · droidcon New York 2024