AI Driven Mobile Testing
summary
An open‑source framework called Trailblaze runs an on‑device LLM loop (OpenAI or Gemini via Kook) to convert natural‑language test objectives into type‑safe tool calls that map to Maestro on-device commands; the prototype is ~4–500 lines (about 200 lines of system prompt), built in six to seven months, and debugs via a log server. Demos are flaky but core approach works: 41 GitHub stars so far, aiming for 100.
AI-written summary · about
chapters (22)
- 0:00AI-driven mobile testing intro
- 1:04Project origins and open source
- 5:13Prototype and inspiration
- 7:23Manual testing challenges at Cash App
- 8:26AI testing goals defined
- 9:08First AI testing prototype
- 14:22Team expansion and open source
- 15:27Scaling AI tests in production
- 17:12UI testing interaction methods
- 18:35Maestro for on-device control
- 20:57Type-safe AI interaction tools
- 22:23LLM debugging challenges
- 24:07AI test execution logging
- 25:11Handling LLM tool call failures
- 27:17UI element selection issues
- 28:40Set-of-mark prompting
- 30:46Trailblaze architecture
- 32:52Future plans and adoption
- 34:17Custom app workflow tools
- 35:40Extending to iOS and web
- 36:22Goose and MCP demo
- 39:57Q&A
related talks
54mAI for Software Testing: The Ultimate Journey
19mThe GPT-ification of Mobile App Testing
11mDragonCrawl: Revolutionizing Mobile Testing with GenAI
35mApp Testing with AI Tools – Make it Easy
20mYour AI Wrote the Code. Now What?
16mYou can’t write “AI” without an “I” Control and responsibility in the age of copilots
26mAI and the Future of Testing
55mAI-First Testing