Stop Guessing and Start Measuring - Benchmarking in Practice
summary
Benchmarks using benchee on Elixir and Phoenix produce concrete surprises: argument order and tail recursion change speed and memory, a real bug shows a query scanning 2.5 million locations until a combined Postgres index fixes it, and JIT and warmup time alter results by orders of magnitude. The session gives iterations per second, standard deviation, inputs used, and the exact code and measurement setup so viewers learn repeatable fixes, numbers, and tools rather than vague advice.
AI-written summary · about
chapters (46)
- 0:00Intro
- 0:32Repetition Logic
- 1:37The Biggest Success
- 1:58I have no idea
- 4:05Graphs
- 5:07Who am I
- 5:26I am Foxy
- 6:10Real Talk
- 6:57What is benchmarking
- 7:08What benchmarking actually is
- 8:04Profiling vs benchmarking
- 9:06Custom Benchmarks
- 9:17Phoenix Benchmark
- 10:06Run Time
- 10:18How Long
- 10:40Worst Case
- 11:39Other Questions
- 11:50Example
- 13:16Sorting and then reversing
- 14:11Premature optimization
- 14:22Not reading the sources
- 14:34Efficiency
- 15:03Marginal
- 15:24Benchmarks
- 16:07Benchmarking Pyramid
- 16:46Benchmark Properties
- 17:11Benchmark Complexity
- 19:31What Good Benchmarking Looks Like
- 20:08System Specification
- 20:48Bench
- 21:02Benchmark Examples
- 22:07Corrected Meaningful Setup
- 22:53Dont Have Select Running
- 23:22Logging
- 23:57Garbage Collection
- 24:15Garbage Collection Graph
- 24:57Warmup
- 25:40Time
- 28:18Matter
- 28:46Polish
- 29:05Alex
- 38:06Results
- 38:36Optimization
- 39:09Performance Graphs
- 41:49Run your own benchmarks
- 42:44Questions
related talks
43mStop Guessing and Start Measuring - Benchmarking in Practice
31mMaking it Fast: The Power of Benchmarking
02h 44mMeasure, don’t guess: benchmarking stories from the trenches
27mOn Benchmarking With Ruby
06mIGNITE: How to benchmark … poorly
21mBenchmarking Like a Boss
35mBenchmarking: You're Doing It Wrong
38mYour benchmark may not guide a real application performance