AI-Driven Load Test Analysis: From Routine Toward Action
summary
Delivery Hero runs ~11 million orders/day and enforces a 3x multiplier requirement for weekly load tests; an automated Go-templated reporter pulls Prometheus and CloudWatch metrics, then an AI pipeline evaluates definitions, production vs test data, and formatted responses. Engineering moves metrics into CSV payloads for AI to avoid context-window noise; the system flags issues like identified pods hitting 100% memory, suggests investigating memory leaks and CPU limits, and reduces manual review, though model drift and hallucinations require prompt/version management.
AI-written summary · about
chapters (13)
- 0:00AI-driven load test analysis intro
- 1:44Reliability manifesto challenges
- 3:31Manual load test reporting issues
- 4:16Automating reports with Go templates
- 6:04Need for AI in reporting
- 7:09Why traditional AI failed
- 10:24Structuring AI prompts
- 11:54AI prompt design components
- 13:16Handling massive data limits
- 16:33Breakthrough with structured CSV
- 19:04Structured data transformed AI
- 21:39AI insights from load tests
- 22:00Final thoughts and opportunities
related talks
37mExploratory Load-Testing with AI Tooling - it's Time to Ramp Up!
51mAI-Driven Strategies for Scalable & Resilient Performance Engg.
42mIt's Inevitable, Mr. Anderson: Machines, Humans, and the Future of Testing
01h 04mImpact of AI on #SoftwareTesting: Are Testers Ready? 🔮
43mAI-Driven Quality Engineering Practices
41mAlexander Gyulai: A Practical Guide to Transforming Quality Assurance with Intelligent Solutions
47mNext generation of API testing : Unleashing the power of AI
34mThe Hard Truth About AI Agents in Production