UTFix: Change Aware Unit Test Repairing using LLM
summary
UTFix uses LLM prompts plus static and dynamic code slicing to repair test assertions and restore coverage, taking original and updated focal methods, the failing test and failure log as inputs. It repairs 89.2% assertion failures on a synthetic Tool-Bench and achieves 100% coverage for 96/369 tests, and on real-world data repairs 60% of failures and gets 100% coverage for 19/30 tests. The system validates by executing generated tests and iterates with minimized traces.
AI-written summary · about
chapters (9)
- 0:00UTFix introduction and problem
- 1:46Outdated test failure example
- 3:09LLM-generated updated tests
- 4:14Validator for test correctness
- 4:57Context gathering with slicing
- 6:03Dataset for evaluation
- 9:36UTFix effectiveness and runtime
- 11:03Conclusion and future work
- 11:25Q&A
related talks
27mQuest for Adequate Autotest Coverage
28mFixit linter+AI coding
14mRIFF: Reduced Instruction Footprint for Coverage-Guided Fuzzing
13mUpFuzz: Detecting Data Format Incompatibility Bugs during Distributed Storage System...
19mCAFault: Enhance Fault Injection Technique in Practical Distributed Systems via...
18mOptimization-Directed Compiler Fuzzing for Continuous Translation Validation
24mGain more confidence in your code with mutation testing
26mDifferential Testing with Foundry