TestMu Conference 2025
as the organisers put it
TestMu Conference is a virtual or online-only conference to define the future of testing. Join over 50,000+ software testers, developers, quality assurance experts, industry experts, and thought leaders for 3 days of learning, testing, and networking at Testμ Conference 2025 by TestMu AI (Formerly LambdaTest). Attend 80+ sessions, workshops, and tutorials, and learn from 100+ speakers. Register for free now | Attend from 19-21 August 2025
who it's for
quality engineers, test automation engineers, qa managers, engineering managers, devops engineers, accessibility specialists
topics
agentic AI for testing; model context protocol (MCP); retrieval augmented generation (RAG) systems; Playwright with AI; Appium 3 architecture; flaky test detection lifecycle; DORA metrics for delivery; automated accessibility audits (axe, Accessibility Insights); scalable CI for thousands of tests; evaluation-driven agent development; MLflow 3 and OpenTelemetry instrumentation
| title | author | length |
|---|---|---|
| Welcome Note - TestMu 2025 | Joe Colantonio | 16m |
Opening salvo lists concrete scale and logistics: 70,000+ registrants and 75+ sessions with 100+ speakers on Lambda Test, plus recorded sessions. Agenda… | ||
| Keynote: Air Fryers, Automation, and AI | Angie Jones | 33m |
Connects air fryer success with concrete automation outcomes: an AI agent called Codeame Goose uses browser drivers (Selenium/Playwright via model context… | ||
| Intent Over Scripts: Modernizing Software Testing with AI | Mayank Bhola, Shantanu Wali, Prince Verma | 34m |
Claims a move from scripts to intent-over-scripts with KAI generating and validating tests end-to-end: 150+ customers, >500 test cases created daily and >300… | ||
| CI and the Great Flakiness Adventure | Simon Mavi Stewart | 31m |
Defines intrarun vs interrun flakiness and a four-step lifecycle: detect assign resolve verify. Recommends automated cron jobs that pull historical results… | ||
| AI for Accessibility: Empowering Inclusive Digital Experiences | Laveena Ramchandani | 34m |
AI can automatically flag missing alt text, bad contrast and navigation barriers and plug into CI pipelines using tools like axe and Accessibility Insights… | ||
| Panel Discussion: AI and Community: Shifting Roles, Rising Impact | Daniel Roe, Karan M V, Maaret Pyhäjärvi, Ram Iyengar | 46m |
Panelists argue AI drives up pull request volume and yields many low-quality PRs unless maintainers add gatekeeping, citing cases where LLMs create failing… | ||
| Ask Me Anything: Future-Proof Your Career: AI, Testing & Path Ahead | Rahul Shetty | 42m |
AI already reshapes QA with realistic forecasts of 10–20% headcount cuts and a clear path for those who upskill: keep core testing (logic, API, SQL) and add… | ||
| The Agentic Shift- Are We Reasoning, Or Just Retrieving Smarter? | Jaydeep Chakrabarty | 52m |
Argues that 2025 is the era of agents not magic: RAG systems give context while agentic frameworks (using tool APIs and ReAct patterns) generate their own… | ||
| Accelerating Success: How to Optimize Value Delivery with DORA | Nathen Harvey | 28m |
DORA defines measurable change enablement with 4 DORA metrics — change lead time, deployment frequency, change failure rate and time to recover — and shows… | ||
| Rapid Threat-to-Test for Agents | Harinee Muralinath | 53m |
Maps agent risks to a concrete Maestro seven-layer breakdown (foundation models, data ops, frameworks, deployment, observability, compliance, ecosystem plus… | ||
| How to Build Enterprise-Grade AI Agents Using Robust Evaluation | Viktoria Semaan | 47m |
Shows a concrete pipeline: instrumented agents with MLflow 3 and ML tracing (OpenTelemetry), a vector database retriever, and tool calls logged so tokens… | ||
| Ship Code. Without Writing It. | Keerthivasan Devaraj, Gurleen Kaur | 55m |
Demoed system takes a Jira ticket, reads project code with client, consults custom MCP protocol servers for Jira/Git/Confluence context, and raises a branch +… | ||
| Exploratory Testing with AI | Gil Zilberfeld | 43m |
AI generates focused session-based testing charters and prioritized test lists in seconds, uses Claude to suggest API and UI charters, and auto-creates JSON… | ||
| AI-Powered Debugging & Browser Automation with Playwright MCP | Debbie O'Brien | 45m |
Playwright MCP generates end to end tests from prompts, runs them, and reports results with roughly 50/50 initial pass rate during the demo; the agent explores… | ||
| Network Control for End-to-End Web Testing | Gleb Bahmutov | 45m |
Plugin Cypress Magic Backend records requests and responses to JSON, replays them to intercept XHRs and cut a 6s test down to roughly 600ms — a 10x speedup… | ||
| Keynote: Architecting Systems for Agent Experience (AX) | Dana Lawson | 32m |
Netlify reports 38% of deploys coming from bots and one internal agent fixes code and merges in 27 seconds, illustrating agent-first flows. The talk shows… | ||
| So you think a new tool will help? Here’s an idea-t to think about… | Isabel Evans | 50m |
A people-first decision framework called IDEA-T presents 12 concrete heuristics, linked to quality attributes and activities in a GitHub repo under GPL 3.0… | ||
| AI, Automation & DevEx: Fueling High-Velocity Engineering | Prashant Sharma | 47m |
Argues developer experience shifts from docs and fast builds to measurable developer flow and claims velocity is now survival, not just advantage. Cites tools… | ||
| Fast Doesn’t Mean Fragile: Delivering AI-Powered Software at Scale | Rob Zuber | 50m |
CircleCI runs 7 million builds/month across ~200 engineers and uses DORA metrics to quantify delivery: a sample elite profile at three deploys/day, 45 minute… | ||
| How to Test LLM Agents | James Massa | 50m |
Claims concrete wins: enterprise migration of 50+ apps to AWS, a generative DVI platform (Complhawk) that cuts manual analysis by 99.98% and saves ~$25M… | ||
| How AI is Exposing the Existential Crisis of Software Testing | Lalitkumar, Udita | 44m |
Claim: AI generates tests and scripts at scale (thousands to millions), exposing that traditional test case counting is commoditized and noise. Practical… | ||
| Your Test Suite Can’t Catch a Hallucination: Talk on AI in Production | Rashi Agrawal | 47m |
A concrete failure mode: an agent told a borrower their loan was declined for low credit instead of missing proof of income, because outputs blend database… | ||
| Event Driven Architecture: Love Triangle in Integration Testing | Andres Damian Sacco | 49m |
Shows a concrete test setup that spins Docker containers with MongoDB and LocalStack + SQS, runs the Java service, publishes a payment event, and verifies DB… | ||
| When Life Gives You Lemons, Are You Counting Them or Making Lemonade? | Chris Armstrong | 48m |
Argues that common KPIs like tests executed or bugs reported mislead decision making and offers a practical 7-step framework to design value-focused… | ||
| AI-Driven Quality Engineering Practices | Niranjani Manoharan | 43m |
An AWS event‑driven pipeline combines CI/CD and CloudWatch logs with Lambda + SageMaker to cluster error messages and predict impact, running as an MVP for… | ||
| Panel Discussion: Transforming Retail with Quality Engineering | Dayapreet Singh, Impu Chunchegowda, Jason Bryant, Shashi Ereti | 01h 00m |
Panelists describe concrete wins: a goal of >80% automation at Dollar General, CI/CD security scans with SonarQube and Veracode that create immediate tickets… | ||
| Panel Discussion: Role of QE in Shaping the Future of Financial Services | Preetham Sunilkumar, Shiva Krishna Kodithyala, Sudeepta Guchhait | 01h 01m |
Panelists report concrete practices: teams integrate continuous testing with real‑time monitoring, shift left into the PDLC, and run CI/CD pipelines using… | ||
| Opening Note - Day 2 | Rosie Sherry | 15m |
Ministry of Testing grows from 18 years of work and 15+ years of community focus into a global platform that scaled to millions of users, and the conference… | ||
| Workshop: Build Your Testing Sidekick- Custom Tools with Model Context Protocol | Sai Krishna, Srinivasan Sekar | 01h 29m |
Builds a TypeScript MCP server that registers a browser metrics collector tool and reuses Playwright MCP for navigation via a CDP endpoint (remote debugging… | ||
| Workshop: Reactive Browser Testing with WebDriver BiDi | Sri Harsha, Maksim Sadym | 01h 26m |
Hands‑on workshop shows WebDriver BiDi used with Selenium 4.35 and Java (Maven) to open a websocket (capability websocketURL=true), subscribe to browser… | ||
| Keynote: How Software Testing Can Increase Agent Autonomy | Luis Héctor Chávez | 31m |
Replit shows an agent that generates full‑stack apps, runs deterministic checks (lint, unit, security scans before production) and uses Playwright API plus a… | ||
| Panel Discussion: What Can Go Wrong With AI in Testing? | Benjamin Bischoff, Brijesh Deb, James Bach, Michael Bolton, Seema Prabhu | 50m |
Panel covers concrete failures and experiments: AI agents deleting code, prompt injection and data leakage, Copilot-style edits that introduce fake functions… | ||
| Code It Forward: Making Your Mark in Open Source | Pallavi Sharma | 31m |
96–97% of modern apps and codebases include open‑source components and a Log4Shell‑style bug once affected ~3 billion devices, so contributing fixes matters… | ||
| Introducing Agent Testing | Sai Krishna, Srinivasan Sekar | 31m |
TestMu AI shows a beta agent-to-agent testing platform that ingests PRDs, Confluence and Jira, auto-generates hundreds of scenarios (intent recognition… | ||
| Balancing Early Testing with Real-World Reliability | Ashish Ghosh | 34m |
Argues that five concrete quality anchors map to SDLC stages and must be enforced: risk‑based acceptance criteria at define, heavy unit tests and contract… | ||
| Panel Discussion: Strategies for Scaling AI in Quality Engineering | Dror Avrilingi, Khimanand Upreti, Janakiraman Jayachandran, Mobin Thomas, Vikul Gupta | 59m |
Panelists present a practical playbook: enterprises report 97.5% daily AI adoption but face blocked scale from data silos, inadequate infra (only ~15–20%… | ||
| Advanced Playwright with AI | Andrew Knight | 01h 31m |
Shows how Playwright plus AI actually produces runnable artifacts: an AI-generated project scaffold (Node/TypeScript, React or Vue front end, Express + SQLite… | ||
| Generative to Agentic to Quantum - The Evolution of AI | Siddhant Wadhwani | 54m |
Maps generative → agentic → quantum with concrete claims: enterprise pilots cut scoped QA and dev effort up to 70%, foundation models handle multi‑turn… | ||
| Test Data Key to Effective Test Coverage | Ashok Kumar | 47m |
Argues that test data is the lever that raises real coverage and shows a working TDM framework that creates data via repurposed automation scripts and… | ||
| Panel Discussion: QE Strategic Shift: What's Changing with AI, Automation, and Speed? | Ayoola Ogunsola, Bhupesh Mittal, Navneet Goyal, Trincy Thomas, Zeba Khan | 47m |
Teams report concrete changes: adoption of Microsoft Copilot inside Visual Studio Code to generate test data and review PRs, continued Selenium + Java suites… | ||
| Building AI Fluency: Leading Teams Through the Learning Curve | Kelly Vaughn | 45m |
Argues that AI fluency is judgment not tools and introduces the PATH framework: Psychological safety, Adopt gradually, Teach through champions, Harmonize… | ||
| The Practical Automation Playbook | Amanda Curtis | 46m |
A five-step playbook (pick goals, resources, tests, increments, metrics) shows how to make automation fast and sustainable: pick business outcomes not test… | ||
| Building Handwriting Recognition System for the NY Times Crossword | Shafik Quoraishee | 44m |
A prototype handwriting recognizer for the New York Times Crossword emerges from an internal Maker Week hackathon and runs entirely on Android with an… | ||
| Keynote: Agentic Cloud: Using Agents to Build Tomorrow’s Cloud | Bratin Saha | 32m |
DigitalOcean describes an end-to-end Gradient AI platform that just hit GA and already hosts 14,000 agents, several thousand in production and billions of… | ||
| QA to QE: Scaling Quality with Ownership, Code, and Culture | Natalie Cole | 46m |
QA practice moves to QE by embedding ownership, automation, and metrics across the delivery pipeline; an automation audit, added unit tests, API slicing… | ||
| Automated Test Data Portal for Financial Services | Rakesh Sukla | 46m |
A banking-grade self-service portal is described that automates creation, provisioning and lifecycle of test data across cloud, mainframe and third-party… | ||
| Enhancing Agent Reliability Through Evaluation-Driven Development | Shadab Nazar | 46m |
Claim: roughly 95% fail-to-prod for AI projects. Presents an actionable evaluation loop: build small manual datasets, augment with synthetic LLM-generated… | ||
| Ensuring Quality Testing in an AI-driven World | Michelle Duke | 52m |
Demo shows a real repo (Mish Friendly Food) getting a new export feature with four card styles, implemented and iterated by GitHub Copilot agent mode, which… | ||
| AI-Driven Strategies for Scalable & Resilient Performance Engg. | Jerome Antonisamy | 51m |
Claims an AI Accelerated PE pipeline that auto-ingests NFRs, generates JMeter/LoadRunner scripts, analyses APM data (Splunk, Grafana, NewRelic, Dynatrace… | ||
| Opening Note - Day 3 | Sanjay Kumar | 15m |
SelectorsHub grows from two early tools (Selector Server and TestCase Studio) into 8 free testing tools used by 1.5 million testers worldwide, built without… | ||
| Workshop: Mastering Appium 3: Architecture, Gestures & Beyond | Sai Krishna, Srinivasan Sekar | 01h 31m |
Appium 3 ships as a modular, pluginable server (released two days before the workshop) that requires Node >=20.19 and npm >=10, moves inspector into a plugin… | ||
| Workshop: Automating Test Environments for DevOps & QA | Mumshad Mannambeth | 01h 32m |
Demonstrates Model Context Protocol as a concrete layer that turns LLM agents into real automation: a local Python MCP using fastmcp exposes resources, tools… | ||
| Keynote: AI & GenAI in Quality Engineering: Crawl, Walk, Run | Subba Ramaswamy | 32m |
Roadmap for adopting AI in QA with concrete wins and examples starting from generation to agentic orchestration including self-healing automation and coverage… | ||
| Panel Discussion: Embracing Agentic AI: From Autonomous Goals to Enterprise Guarantees | Anjali Chhabra, Hemalatha Murugesan, Neelmani Verma, Nilesh Yeola, Prasad Siddaiah, Rutvik Mrug | 01h 02m |
Panel lists concrete wins and gaps: companies report 25–30% agentic workflows today, deploying impact‑based regression selection, RCA and defect‑triage agents… | ||
| How to Get AI to Stop Being Weird and Actually Be Useful | Dona Sarkar | 36m |
Claims that AI does “30–50% of work” and that it will wipe out entry‑level jobs are overstated; concrete failure modes are hallucinations and agents drifting… | ||
| Should We Let AI Take Over Test Automation Completely? | Ken S'ng Wong | 40m |
Covers concrete experiments: compares QP3D vs WebGL screenshots, uses image segmentation visual testing and perceptual hashing to accept contextual matches… | ||
| From Hours to Minutes: Run Thousands of CI Tests in Just Minute | Pramod Yadav | 33m |
Claims a working tool called RunRight that uses Playwright, GitHub Actions matrices, a custom state.json reporter and pre/post-commit hooks to measure per-test… | ||
| Evaluating RAG Applications: From Retrieval to Response Quality | Srinivas Chitta | 31m |
Demo shows evaluating RAG apps end-to-end with RAGAS: retrieve snippets from embeddings and a vector DB, measure context precision and context recall plus… | ||
| Stop Breaking Your Teams: Seeing the Whole Instead of Pieces | Nithin S.S | 31m |
Argues that testing must move from isolated feature checks to a customer-centered experience strategy, shows how Spotify-style squads create silos that reduce… | ||
| Surviving and Thriving with AI in QA | Maryia Tuleika | 41m |
Cuts the AI hype with specific limits and tactics: warns of model collapse and LLMs lacking spatial understanding and determinism, shows a working custom… | ||
| The Quality Leadership Shift: From Compulsiveness to Cautiousness | Sachin Sharma | 38m |
Argues that quality leadership must move from compulsive checklist testing to value-driven cautiousness, using AI as a toolbox not just ChatGPT — repeated… | ||
| Full Court Quality: Lacing Up for Speed, Stability & Style | Sunita McCoy | 46m |
Presents a repeatable enterprise framework that unifies quality engineering, developer experience and observability to increase velocity while cutting defects… | ||
| Navigating Mobile App Testing and App Store Rejection | Saif Siddiqui | 42m |
Covers concrete tactics: run functional, performance, security, compatibility and usability tests, use TestFlight and App Store Connect plus Xcode for binary… | ||
| Randomized testing: Gotta Catch ‘Em All | David Pilato | 45m |
Shows that adding randomness finds bugs missed by hand: uses the Carrot Search randomized runner (Carrot Search) with JUnit4 to vary locales, time zones… | ||
| Balancing Release & Sprint Delivery Speed with Thorough Testing | Jim Trentadue | 49m |
Practical checklist for tonight: AI generates edge, negative and boundary test ideas from user stories and produces draft test steps and defect template shells… | ||
| Keynote: Building for AI at Scale: Infra, Integrity, & Innovation | T.R. Vishwanath | 34m |
Glean connects Slack Jira Google Docs Salesforce and other silos into a single index with user-aware permission enforcement so answers map back to source… | ||
| Panel Discussion: Building Confidence in AI-Driven Testing Decisions | Asaf Saar, 𝐂𝐨𝐭𝐲 𝐑𝐨𝐬𝐞𝐧𝐛𝐥𝐚𝐭𝐡, Guljeet Nagpaul, Idan Gazit | 01h 05m |
Panelists lay out concrete practices: use RAG and model-versioning for data security, track provenance and audit trails for every AI-generated test, run eval… | ||
| Observability - Holistic Quality across Software Systems | Sidney Karoffa | 47m |
Walks through building observability as a quality practice using Datadog dashboards, a company training, and a recurring monthly infra ops meeting that… | ||
| From SDLC to ADLC: The Enterprise Agent Development Lifecycle | Sam Julien | 43m |
Enterprises must adopt an Agent Development Life Cycle that layers on SDLC and prioritizes outcomes, modular design and evals over traditional QA. Built-in… | ||
| Agentic Testing: Your Skilled Human Tester | Gregory Goldshteyn | 47m |
Agentic testing uses LLMs (Gemini, GPT) plus tools like Playwright MCP, Postman agents and Mabel to run goal‑driven, multi‑agent tests that observe DOM and… | ||
| Evolution of Quality Engineering in Financial Services | Preetham Sunilkumar | 47m |
Claims that AI is non‑negotiable for financial QE and that four agent types can move testing from commits to production: a commit agent that summarizes code… | ||
| Closing Note | Mudit Singh, Kavya Nair, Sparsh Kesari | 19m |
Conference reflects four editions since 2022 and reports 75,000+ registrations, 105 speakers and over 3,360 minutes of sessions, with 63 partners, 28% women… | ||
| nothing matches | ||