Keynote: How Software Testing Can Increase Agent Autonomy
summary
Replit shows an agent that generates full‑stack apps, runs deterministic checks (lint, unit, security scans before production) and uses Playwright API plus a vision tool to take screenshots and query the DOM. Benchmarks show pure code repair leaders at SWE-bench 75% success while visual multimodal tasks sit around multimodal 30–35%. Incremental test synthesis and reuse of Playwright locators lets the agent run and fix end‑to‑end failures, increasing autonomy up to about 3x in internal tests.
AI-written summary · about
chapters (15)
- 0:00Conference opening
- 1:32AI agent systems and testing
- 5:02What is an AI agent
- 8:12Context window limits
- 11:01Testing agent feedback loops
- 13:51Traditional tests in multimodal
- 14:35Replet’s AI-powered web app
- 16:43Planning and human-in-the-loop
- 18:11Automated versus contextual errors
- 20:40Security testing trade-offs
- 22:07Multimodal testing challenges
- 23:52Automated end-to-end testing
- 27:00Incremental testing with Playwright
- 29:10Results and future of testing
- 29:53Key takeaways
related talks
25mThe 3 Pillars of Autonomy
55mKeynote: Who are we Now?
48mSupercharged Testing: AI-Powered Workflows with Playwright + MCP
14mAutonomous Testing Agents with LLMs in React
55mKeynote: Exoskeletons, not Autopilots
54mKeynote: AI-Powered App Development
22mKeynote
18mReinventing App Testing with Agentic Apps