Honest Performance Testing
summary
The talk explains why standard statistical metrics (means, medians) are misleading for performance testing of applications and argues that average response time is meaningless. It demonstrates techniques to address this—showing how to avoid deceptive aggregate metrics—and proposes a practical alternative statistical approach validated in real use.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 2:07Problem relevance
- 4:16Testing goal
- 6:20Meaningless averages
- 9:34Median and percentiles
- 15:17Why averages fail
- 17:25Working with percentiles
- 22:49Trend analysis over numbers
- 27:09How to get profiles
- 30:45Summary
- 31:53Q&A
related talks
49mPerformance Testing
54mPerformance tests and other lies
47mLee Barnes: Maximizing the Value of Performance Testing Throughout the Software Development Cycle
12mPerformance In Context
40mPerformance Testing Done Right
06mMeasurement is Not Enough: Rise of Performance Analytics
57mOpen Forum on Performance Testing
42mMeasuring Performance with Functional Tests: A Low-Effort Approach for High Impact
