TestMu Conference 2026
| title | author | length |
|---|---|---|
| Welcome to TestMu Conference 2026 | Joe Colantonio | 14m |
75,000 registrants and a three‑day schedule with 80+ sessions lay out practical coverage: agentic AI workflows, how to validate and put guardrails on agents… | ||
| Keynote: Testing In The Era of AI - From QA to Trust Engineering | Kishor Wikhe | 20m |
Claims that testing shifts from finding defects to building trust engineering, showing AI QA must cover reliability, security, governance, observability and… | ||
| Building Deterministic Infrastructure for Non Deterministic AI Agents | Nishant Gupta | 20m |
Claims that production must wrap LLMs with a deterministic execution shell: orchestration owns retries, validation, and idempotent execution; tools expose JSON… | ||
| Confidence - Correctness: The Agentic Validation Loop | Prince Verma | 36m |
Claim: agentic pipelines grade their own work so confidence often equals false certainty. The talk demos a closed-loop Agentic Validation Loop and an external… | ||
| Ask Me Anything: AI Agents, Skills, Career & Growth | Rahul Shetty | 33m |
Agentic AI is already writing code, generating and triaging tests and the practical advice is clear: learn fundamentals first, then Playwright + TypeScript for… | ||
| Context That Dreams | Jaydeep Chakrabarty | 40m |
Claims a ceiling on filing-cabinet context layers and demonstrates a working alternative: a pipeline that pulls Zoom, Outlook, Jira and Confluence into a… | ||
| From AI Assistants to AI Coworkers: How Engineering Teams Ship Faster with Enterprise Context | Nilesh Dalvi | 27m |
Engineering coding ramps 5x faster but the full delivery cycle only drops 16% because 20 of 100 hours are coding; the work instead floods into discovery, eval… | ||
| Architects of Trust: Redefining Quality Leadership in an Agentic World | Sobhitha Neelanath | 44m |
Classical coverage metrics (94.2% test coverage) hide a midnight recursive prompt loop that costs $400,000 before alerts fire, so trust becomes a measurable… | ||
| When Software Becomes Everyone's Job | Gurleen Kaur, Keerthivasan Devaraj | 46m |
Organization-wide shift from IDE plugins to a platform changes who builds and who governs: a 48‑hour Tatkal squad proves many features can reach production… | ||
| Fireside Chat: AI Trust and Governance in QE - Who Actually Signs Off | 𝐇𝐚𝐫𝐥𝐞𝐞𝐧 𝐁𝐞𝐝𝐢 | 32m |
Argues that QE must own an accountability layer as teams ship code that AI writes and runs tests, showing why governance either becomes compliance theater or… | ||
| Can You Trust Your ChatBot: Techniques for Testing LLM Responses | Gil Zilberfeld | 47m |
Builds a Python test planning agent that sends prompts to Gemini, returns structured API test plans, and iterates using human golden datasets and automated… | ||
| Flaky, Fragile, and Forgotten: The Fall of a Test Automation Project | Mesut Durukal | 46m |
A real postmortem of a repo that becomes the team blocker: flaky UI locators break after one library change, a CI suite grows to 500 tests and a 4-hour… | ||
| The Right Model for the Right Job: Cutting LLM Costs Without Cutting Quality | Viktoria Semaan | 33m |
Demo moderates high volume soccer chat with an 8‑label classification set and runs a 16‑model benchmark to pick the cheapest model that meets a quality bar… | ||
| System Prompt Design for Voice Agents | Amanda Martin | 30m |
Claim: system prompts change behavior dramatically — Vapi analyzes over a million calls and runs a live demo comparing three prompt strategies using OpenAI… | ||
| Testing at the Speed of Agents: Salesforce's Evolving Approach to Agentic Testing | John Liang | 33m |
Salesforce outlines a phased QA shift: validate intermediate traces (utterances, tool calls, grounding) not just final responses; build industry-specific test… | ||
| The Bionic Workforce: Where Humans and AI Co-exist | Lisha Rakesh | 47m |
Claims that quality engineering shifts across assisted, augmented, autonomous modes with concrete metrics: about 90% of orgs piloting GenAI, 68% using it for… | ||
| Real World Lessons from Scaling Test Automation | Deepak Kamboj | 49m |
Describes running Playwright at enterprise scale across a monorepo with ~14,000 tests, pairing Playwright MCP and CI (Azure DevOps/GitHub) with… | ||
| Why RL Environments Are All You Need for Building Agents | Mahesh Sathiamoorthy | 44m |
Argues that RL environments are the critical ingredient: Bespoke Labs curates sandboxed environments and verifiers, runs SFT and RL and uses prompt evolution… | ||
| Build Trustworthy AI Agents powered by Evals | Rushabh Mehta | 47m |
Asserts that LGTM is not a deployment strategy and shows concrete eval practices: use deterministic code graders plus LLM-as-judge and human graders, run… | ||
| Canal Mania and the Philosopher's Stone | Magnus K. Hillestad | 35m |
Claims that the AI revolution has barely begun and that what’s missing is an OS for intelligence, not just bigger models. Sanity prototypes a team-of-agents… | ||
| Panel Discussion: The BFSI Playbook - Scaling AI with Trust and Quality | Eshwar Pastapur, Brianna Elsass, Brian Corkery, Anand Moorthy | 44m |
Banks and insurers are moving a few pilot projects into production by starting with low‑risk workflows like summarizing claim files and automating test script… | ||
| Backwards Scoring: Ranking Test Suites by Which Real Incidents They Would Have Caught | Partha Sarathi Samal | 33m |
A method replays each logged incident against the system state just before its first symptom so every test gets a verdict by failure history: Record, Revert… | ||
| Beyond Scoring: Rethinking Ranking in the LLM Era | Rhea Goel | 34m |
Modern ranking swaps handcrafted objective weights for natural language objectives and shows how to combine reasoning with behavioral prediction: start with… | ||
| Testing What Matters: Evaluating LLM Relevancy with DeepEval | Monika Sharma | 42m |
DeepEval is a Python library that runs LLM test cases through a judge model using the G-Eval technique, installs via UV/virtualenv and calls model APIs (demo… | ||
| Working With Developers: A Tester's Survival Guide | Wessel Braakman | 41m |
Argues that a tester’s success rests on adapting communication to developer personalities and shows concrete before/after scripts: leading with impacts 30% of… | ||
| Panel Discussion: QE at Enterprise Scale - What Will a QE Org Look Like in 2027? | Impu Chunchegowda, Senthil Kannan, 𝐓𝐨𝐝𝐝 𝐋𝐞𝐦𝐦𝐨𝐧𝐝𝐬 | 49m |
Claims that AI agents produce test suites, run pipelines and triage bugs are already live: teams train an in-house code review agent, shift from embedded QA to… | ||
| Keynote: Is the Developer Lifecycle Dead? | Thomas Dohmke | 41m |
Argues that agentic development produces far more code and faster — agents run parallel threads overnight, founders set 2 AM alarms when loops stall, and teams… | ||
| You Can't assertEquals an Agent: A Tester's Guide to Agentic Quality | Gaurav Khurana | 54m |
Shows that assertEquals fails for agentic workflows and gives a five‑layer playbook implemented in demo code: a refund bot that logs tool calls and a refund… | ||
| Your AI Agent Passed Every Test. Your Humans Still Rejected It. Now What? | Jill Hosmer-Jolley | 01h 02m |
When agents pass technical tests but users still reject them, the missing work is the human quality layer: measurable visibility, reproducible artifacts, and… | ||
| Stop Guessing A11y: Auto-Generate Playwright Tests from Your GraphQL Schema | Vanessa Johnson | 31m |
Claim: add lightweight @a11y hints (directives: label, role, template, token) to a GraphQL schema, run a single-file Node generator that emits a JSON spec and… | ||
| Panel Discussion: Scaling Enterprise Practice in the Agentic Era | Mhahesh Muraleedhara, Dror Avrilingi, Vikul Gupta, 𝐑𝐢𝐜𝐡𝐚 𝐀𝐠𝐫𝐚𝐰𝐚𝐥, Nagendra BS | 50m |
Enterprise pilots that looked impressive often fail to scale because operational details break first: one case creates 1,500 epics and 15,000 user stories plus… | ||
| Why Does AI Suddenly Need Forward-Deployed Engineers? | Harinee Muralinath | 31m |
Claims that AI makes forward-deployed engineering affordable at scale by letting AI collapse execution cost while the cost of human judgment stays high, with… | ||
| The Trust Problem: Designing Quality Frameworks for AI-Generated Code | Neelmani Verma | 26m |
AI-generated code often compiles and passes its own tests but fails intent checks — a third was confidently wrong on first pass; a four-line loyalty function… | ||
| Agentic Engineering: From Writing Code to Reviewing It | Sumit Shatwara | 46m |
Describes an operating model where AI agents accept a ticket and deliver a plan, tests and a reviewable PR rather than a keystroke; a demo shows a PR merged in… | ||
| Going Past the Vibes: How to Build Real Apps | Lazar Jovanovic | 45m |
Claims building 500+ production apps and surviving 37,385 edits backs a workflow that stops re‑prompting and uses concrete features: skills as reusable… | ||
| Bringing Mobile App Validation into the Agentic Loop | Shantanu Wali | 46m |
Every step is concrete: an agentic CLI called Kane CLI drives iOS simulators and Android emulators from the terminal, navigates hybrid native/WebView flows… | ||
| Workshop: Advanced Web Testing with Playwright and AI | Andrew Knight | 01h 31m |
A build-along that actually shows the workflow: Playwright + local CLI skills (not MCP) set up from scratch, Chromium-only config, and a live run against the… | ||
| Panel Discussion: AI in Mobile QA - What Actually Works Today? | Jan Schnyder, Nicolas Muntwyler, Justin Yuan | 49m |
Panel reports that nunu.ai runs vision-first agents that stream real devices, tap and swipe like humans, and currently lead public phone-use benchmarks; agents… | ||
| Local Agentic Theory for Accessible Mobile Games | Shafik Quoraishee | 42m |
Architecture puts tiny reinforcement-learning and reasoning agents directly on phones, trading 200 ms cloud latency vs ~3 ms LiteRT to run… | ||
| Panel Discussion: Money Moves at Machine Speed - Trust, Risk, and Quality in Agentic Finance | Trincy Thomas, Preetham Sunilkumar, Veena Setlur, Krystyna Sylyvonchyk | 01h 04m |
Banks and insurers are already running agents that autonomously handle account maintenance (address and beneficiary updates), quietly revert a reversible… | ||
| From Load Testing to Reliability Engineering: Making Performance Testing Predict Production Behavior | Stephan Mousset | 50m |
Performance testing only becomes predictive when it integrates objectives, observability and production feedback rather than producing reports; at scale —… | ||
| Scaling Trust, Not Automation - Rethinking Quality for the AI Era | Walter Zimerman | 34m |
AI systems delegate judgment, so quality becomes measurable trust rather than more automation; real incidents include a 2016 Alexa clip that produced porn… | ||
| Eval-First QA Agents: Testing Streaming Platforms | Gregory Goldshteyn | 46m |
Builds an agentic QA stack for Apple TV, Roku and Fire TV that refuses release until its tests gate it, claiming a 38-tool MCP server and a 43-API helper… | ||
| Agentic AI Changed What We Test, And How We Test It | Pooja Oza | 31m |
Argues that testing must move from exact-match oracles to a bounded behaviour envelope with three gates: goal achieved, limits respected, and decisions… | ||
| Building Once, Running Everywhere: The Future of Portable AI Applications | Prashant Sharma | 32m |
Claims that one binary can run identically everywhere are false; instead the stack uses portable AI runtimes plus hardware abstraction, intelligent compilation… | ||
| Cloud Assumes You Know What a Request Will Cost | Ojus Save | 36m |
Compares two executions of the same agent prompt—investigate a checkout latency spike—and shows a clean path with 5 nodes versus a failed-test path with 7… | ||
| The New Skill Stack for SDETs in the Agentic Era | Adeel Mansoor | 38m |
SDETs already keep automation frameworks, risk models, CI/CD, debugging taxonomies and code review; the talk argues those skills stack into four new, concrete… | ||
| The Economics of AI Agents: How Startups Are Rethinking Value and Monetization | Vibhor Rastogi | 50m |
Argues that agents break the per‑seat SaaS unit by running hundreds or thousands of bots under one login and proposes seat plus capped consumption as the… | ||
| Panel Discussion: Who Tests the Machines? QE Leaders on Quality in the Age of AI-Written Code | Ankur Agarwal, Nandini Srinivasan, Somu Suryanarayanan, Rahul Nethi | 47m |
Panelists report concrete outcomes: startups ship nearly 90% AI-generated code, one org says 60–65%, another 40–50%; mobile automation built with Appium and… | ||
| Agentic AI and the Next Decade of Quality Engineering | 𝐌𝐚𝐥𝐥𝐢𝐤𝐚 𝐅𝐞𝐫𝐧𝐚𝐧𝐝𝐞𝐬 | 37m |
Argues agentic AI moves repetitive QE work to self-generating test suites that design, write and execute tests, convert legacy suites to Playwright/Selenium… | ||
| When You Run Out of Requirements: What Happens When You Go All-In on Agents | Rashi Agrawal | 46m |
An engineering team goes all-in on agents using Claude Code and Cursor plus custom HIPAA-tuned skills and an agent-first architecture; autonomous builds run… | ||
| Keynote: The Next Era of AI: Moving from “Can We Build It?” to “Can We Trust It?” | Andrew Duncan | 38m |
Argues that enterprises face a growing trust gap as capability outpaces confidence, and that pilots do not equal production; recommends continuous end-to-end… | ||
| Panel Discussion: Mission-Critical Priorities in Quality Engineering: The Leader's Playbook | Ali Karnib, Hina Saini, Akshit Lomash, Cem Sever, Kumar Gaurav Jain, Tenny Agustin | 49m |
Bets focus on turning test automation into AI-powered test infrastructure, rebuilding frameworks toward serverless, event-driven pipelines, and democratizing… | ||
| Panel Discussion: Agentic Engineering and Quality in Healthcare | Chad Ullman, Rohit Jaiswal, Mark Kroh | 47m |
Panelists report concrete results: a $250K consultancy scope is replaced by augmented teams, test authoring in the IDE and first‑pass code review save hours… | ||
| Scaling Quality in a Decision Intelligence Platform: The Agentic QA Playbook | Hammad Ahmed | 31m |
Aily Labs rebuilds QA into an always-on agent network that ingests 10,000 test cases, Confluence and Jira docs, and uses Playwright, Appium, Behave, Datadog… | ||
| Panel Discussion: Driving the Agentic Shift in Banking Adoption Governance and Scale | Javier Solano, Mauricio Sánchez, Humberto Henríquez | 49m |
Panel maps a practical playbook: a live deployment that uses agents for deduplicating the customer golden record with a human-in-the-loop second check each… | ||
| Keynote: The Self Driving Company | Luis Héctor Chávez | 15m |
Replit reports engineering output tripling over seven months while MTBF stays flat and MTTM drops; seven agentic loops automate coding, review, maintenance and… | ||
| AI Applications Security Puzzle | Maryia Tuleika | 57m |
Live product failures: assistants obeying injected commands, a vending machine convinced to sell at negative margins, and fine‑tuned APIs hit with thousands of… | ||
| Panel Discussion: Reinventing the QE Practice at Global Scale in Agentic Era | Raghavendra Prasad MG, Khimanand Upreti, Jeba Abraham, Sunie Paul, Ninad P Shirodkar | 01h 04m |
Claims that global QE no longer scales by headcount: platform- and agent-first delivery replaces pyramid staffing and targets up to 50% reduction in people… | ||
| Workshop: The Full Agentic QA Loop for Mobile Applications | Shravan Mahajan | 57m |
Workshop demos an end-to-end mobile QA loop using Kane CLI driving an iOS simulator to ingest a PRD, generate a context graph, and auto-produce tests: it… | ||
| The Copilot Wrote It but Who Saves It: Surviving the AI Code Tsunami | Mathias Strziga | 31m |
AI-generated code arrives in bursts not sprints, so QA must become Validation Pilots with decision templates, not line-by-line reviewers. The AI Development… | ||
| Evidence-Based QA: Proving Quality with Agentic Test Runs and OpenSourcing How We Do It | Anubhav Singhmaar | 32m |
Evidence-cli creates a sealed, versioned .evidence pack per run that contains L0 (run, test file, result) and optional L1 captures (per-step screenshots… | ||
| ML-Driven Test Intelligence at Scale - What Works What Fails and Why It Matters | Tanvi Mittal | 28m |
Claims a production-ready dual‑layer ML pipeline: XGBoost and scikit-learn for test selection plus an LSTM autoencoder (TensorFlow/PyTorch) for anomaly… | ||
| Building a Billion-Dollar Healthcare Product at Zebra Technologies | Spyros Katopodis | 30m |
A nurse-facing wearable called Zebra Companion listens, transcribes ambient conversations, extracts handoff tasks and patches approved updates into EHRs… | ||
| Productize Yourself: AI Proof Yourself | Rahul Parwal | 31m |
Market reality: trillions invested in AI, junior hiring slows for the first time in 25 years, and résumés lose signal value. The claim is concrete: testers… | ||
| AI Architecture Thinking: From Idea to Impact | Safeena Banu | 44m |
A concrete agentic QA stack is built and shown working end-to-end: a feature planner (Opus frontier model) calls a smaller locator inspector (Playwright + MCP… | ||
| Workshop: Legacy vs Autonomous QA Arena - Surviving the AI Driven Quality Evolution | Athresh Guruprakash | 01h 15m |
Claims that 58% of test failures come from timing and locator issues and that these consume 30–40% of QA budget drive the session: legacy Selenium still… | ||
| Keynote: Will the Real Autonomous Agent Please Stand Up | Dona Sarkar | 32m |
Argues that dependable AI is a validation problem, not raw capability, and shows Microsoft shipping Scout autopilot and copilots as examples that run in the… | ||
| Redefining Test Data Strategy in the Gen AI Era | Manideep Singireddy | 38m |
Claims a repeatable AI-driven framework that turns intent into executable scenarios then into exact records, cutting provisioning time and production… | ||
| Build Your Own Benchmark: Why Agents Need Custom Evals and How to Create Them | Haritha Sreedharan Nair | 45m |
Agents become the primary users of products, so public leaderboards mislead: the real question is whether an agent in this harness completes the job. The… | ||
| From Pilots to Practice: Transforming Sales with AI | Meenakshi Singh | 20m |
85% of enterprises run generative AI pilots but under 18% scale them; a successful rollout looks like Morgan Stanley’s staged launch to 16,000 advisors after… | ||
| Panel Discussion: Engineering Trust Quality in AI-Driven Commerce | Sunita McCoy, Nipam Desai, Jason Bryant, Rama Gunjaipalli, Krishna C Gangu | 01h 01m |
Retail teams describe concrete shifts: Super Bowl traffic hits 1.5 million pizzas with about 5,000 orders a minute, peak readiness becomes continuous not a… | ||
| Fast, Smart, and Fragile: Rethinking Testing Leadership in the Age of AI | Laveena Ramchandani | 32m |
AI-generated tests arrive 3x faster test creation and cut scripting by 70%, but a Microsoft study shows Copilot yields ~55% faster coding and speed hides gaps… | ||
| Autonomous Quality Engineering: AI Systems That Generate Execute Heal Learn and Govern Quality | Jyotheeswara Reddy Gottam | 41m |
A framework that actually runs: generative LLMs plus a multi‑agent coordinator use RAG and vector stores to draft tests, mutation testing to prove they would… | ||
| Discovery Found Everything: Your Agent Still Doesnt Know What Matters | Angela Cutulenco | 33m |
An AI service desk analyst agent runs in production, restricted to one incident type on a single service while ServiceNow performs nightly ServiceNow Discovery… | ||
| Testing the Untestable: Turning OWASP AILLM Risks into Practical QA Checks | Thejes Sree Satheesh Kumar | 28m |
Translates OWASP AI/LLM Top 10 into concrete QA checks with a six-step method: pick a risk, describe a bad day, write probes, find the signal, write checks… | ||
| When Software Starts Thinking: The Now of Quality Engineering | Chinmaya Kothari | 41m |
Generative AI breaks the “same input, same output” rule, so quality work shifts into measurable AI Assurance: map each trust attribute to risks, pick metrics… | ||
| AI-Powered Impact-Based Testing for Faster Safer Releases | Srikanta Sahoo, Navneet Goyal | 52m |
Smart Engine analyzes each commit using 20+ signals — structural imports and call graphs, semantic embeddings (BGE‑M3), historical failure patterns, trust… | ||
| Closing Note | 13m | |
75,000 attendees from 130+ countries, 140+ speakers, 95 sessions and 4,180 minutes of content summarize scale; builders and testers share concrete claims —… | ||
| Panel Discussion: The Agentic Software Factory That Enterprises Actually Need | Heena Purohit, Chase Norton, Itamar Friedman, Patrick Vuong | 46m |
Claim: giving engineers the ability to generate 10x more code does not yield 10x value and often shifts the bottleneck to trusting and verifying output… | ||
| Developing AI Agents for Disability and Self Determination: Lessons from Project RAISE | Lisa Dieker | 43m |
Project RAISE builds a narrow agent called ZB that students select in a competition and help train by recording third‑grade language; facial tracking is… | ||
| Workshop: Engineering RemoteXPC: Building Wireless iOS Automation with Appium | Swastik Baranwal, Navin Chandra | 01h 28m |
Workshop shows a Node.js RemoteXPC client (appium-ios-remotexpc) that replaces USB-only iOS automation with a per-device TLS tunnel and virtual TAP, exposing… | ||
| Revisit Old Problems with New Eyes! | Pramod Yadav | 42m |
Argues agents let one engineer build and ship things that used to need deep pockets: Taxly deployed in under two weeks is a VAT‑filing app with ~85% test… | ||
| Workshop: Debug with Appium MCP | Srinivasan Sekar, Sai Krishna | 01h 23m |
Appium is extended into an AI-native testing platform that understands natural language, uses mcp-appium to bridge Claude-style agents and Appium drivers, and… | ||
| The Last Manual Handoff: Redesigning End-to-End Testing in the Age of AI | Sachin Sharma | 55m |
Argues that every testing box is automated but the arrows between them remain manual, shows Main Tester United — a context-aware chain of AI agents that… | ||
| Workshop: The Full Agentic QA Loop for Web Applications | Siddhant Sinha | 53m |
Workshop shows an end-to-end QA pipeline that turns a PRD into plain-English tests, runs them in a real browser, and emits portable proof. The demo uses… | ||
| Securing Agentic AI | Anamika Mukhopadhyay | 48m |
Agents pass functional tests yet delete production data and leak enterprise secrets by following manipulated goals, not by breaking models. Real examples… | ||
| Testing the Future: TimeTravel Simulation for Financial Systems | Rakesh Sukla | 45m |
Claims a system that produces production outcomes by creating an initial state, then advances time day-by-day via a lifecycle-aware Test Data Management Portal… | ||
| Testing the Agents That Look You in the Eye | Sai Krishna, Srinivasan Sekar | 47m |
A testing pipeline that joins any joinable WebRTC URL (Zoom, Google Meet, web WhatsApp) without SDKs mimics a human with a fake camera and mic, generates… | ||
| nothing matches | ||