Stop Guessing and Start Measuring - Benchmarking in Practice
summary
Starts from a bad quick shell check that reports 65 microseconds vs 210 microseconds then shows a proper harness with 5s runs, warmup, medians, histograms and IQRs that reverse that result and prove recursion can be twice as fast. Demonstrates a sorting surprise where sort then reverse wins by ~4x, and a production bug with 2.3 million locations where query plans and a missing composite index cause timeouts. Emphasizes multiple input sizes, GC and environment control, and measuring insertion and runtime tradeoffs.
AI-written summary · about
chapters (12)
- 0:00Start
- 8:45The Difference between Profiling and Benchmark
- 16:46An Application Level Benchmark
- 17:19Properties of these Different Benchmarks
- 19:33What Am I Benchmarking for
- 20:53Create an Interference Free Environment
- 21:46Garbage Collection
- 30:52A Benchmark for Insertion Time
- 32:24Standard Deviation
- 34:23Median
- 35:27Box Plots
- 39:30The Flat Map Vs Mapped or Flattened Benchmark
related talks
46mStop Guessing and Start Measuring - Benchmarking in Practice
02h 44mMeasure, don’t guess: benchmarking stories from the trenches
02h 41mMicro Benchmarking - The Art of Realizing One is Wrong
27mOn Benchmarking With Ruby
35mBenchmarking: You're Doing It Wrong
06mIGNITE: How to benchmark … poorly
31mMaking it Fast: The Power of Benchmarking
38mYour benchmark may not guide a real application performance