A modern testing tale
summary
Attempts to auto‑generate unit tests for a Vet app reminder feature use JetBrains Assistant, GitHub Copilot and ChatGPT and repeatedly fail to produce reliable coverage: the first run yields four tests that mock most collaborators and are coupled to implementation, later attempts produce three test classes covering only happy paths and one error, many generated tests don't compile and use current time so are non‑deterministic; a date‑calculator collaborator lacks critical coverage, so AI helps create examples but cannot replace human validation.
AI-written summary · about
chapters (12)
- 0:00🚀 Introduction: Exploring AI's Role in Unit Testing
- 2:30The Value of Unit Testing in Software Development
- 5:00The Importance of Good Unit Tests: Fast, Isolated, and Repeatable
- 8:00Overview of the "FIRST" Principles for Writing Effective Unit Tests
- 12:00Understanding the "Testata" Framework for Identifying Quality Unit Tests
- 16:00Introducing the Unit Testing Challenge: Creating Reminders in a Vet App
- 20:00The First Attempt: Using AI to Generate Unit Tests for the Reminder Feature
- 22:00Identifying Issues with AI-Generated Tests: Lack of Determinism and Coupling
- 26:00The Second Attempt: Revising the AI-Generated Tests and Addressing Errors
- 30:00The Final Attempt: Analyzing AI's Successes and Failures in Generating Tests
- 32:00Why AI Won't Fully Replace Human Involvement in Unit Testing
- 34:00Conclusion: How AI Can Help, But Humans Must Remain in Control
related talks
26mAI and the Future of Testing
38mNavigating AI assistant development
37mNo More Writing Test Code: Auto Designing and Generating Asynchronous Tests with JetBrains AI Assistant
29mQA vs AI: Friends or Foes?
35mApp Testing with AI Tools – Make it Easy
40mFrom Bugs to Brilliance: Leveraging AI in Modern Software Testing
42mThe Power of Human and AI Collaboration in Software Testing
34mNavigating AI assistant development