Profiling and Benchmarking 101

on YouTube ↗39m

" summary

Explains a concrete workflow: read production metrics (New Relic, Skylight, Scout), profile the problematic action in development, then isolate and micro/macro benchmark the hot spot with real‑like data. Demonstrates tools and tradeoffs — StackProf and ruby-prof for sampling vs tracing, benchmark-ips with rack-mini-profiler and memory_profiler for iterations and allocations — and gives surprising numbers (Apollo-style 13% extra load, 1201/1202 alarms, 90% duty cycles) to show what actually breaks and what fixes measurably work.

AI-written summary · about

" chapters (23)

1. 0:00Intro
2. 0:18CLOUD Pain Free Rails 66 Deployments
3. 1:22FLIGHT PLAN Why Measure? Benchmarking vs. Profiling Profiling Jumpstart Benchmarking Jumpstart
4. 2:11The APOLLO GUIDANCE COMPUTER
5. 5:53BAILOUT No available core sets
6. 8:53APOLLO: the ultimate "TESTING IN PROD"
7. 10:37WE are not COMPILERS
8. 11:11ALL PERFORMANCE WORK WITHOUT MEASUREMENT is PREMATURE
9. 12:17A BENCHMARK is a SYNTHETIC MEASUREMENT of the RESOURCES or TIME consumed by a defined PROCEDURE
10. 13:12A PROFILE is a STEP BY STEP ACCOUNTING of the RELATIVE CONSUMPTION of RESOURCES by a procedure's many SUBROUTINES
11. 16:32PROFILING answers "WHAT'S SLOW?"
12. 17:29PROFILING ENVIRONMENT
13. 20:39CPU WALL
14. 22:09STATISTICAL TRACING
15. 23:20STACKPROF RUBY-PROF
16. 25:56TRY DIFFERENT VIEWS/FORMATS
17. 26:35rack-mini-profiler
18. 26:58MICRO MACRO APP
19. 30:57WARMUP long enough for STABLE RESULTS
20. 32:33BENCHMARK TIME should be long enough FOR LOW VARIANCE
21. 34:00benchmarks
22. 35:08READ PRODUCTION METRICS PROFILE TO FIND HOTSPOTS CREATE BENCHMARK ITERATE DEPLOY
23. 37:07FAIL GRACEFULLY DON'T PREDICT THE FUTURE STUDY AND REPLICATE THE PAST

" related talks

• 51mProduction Profiling: What, Why and HowRichard Warburton · Voxxed Days Minsk 2018
• 29mHow to PerformanceEileen M Uchitelle · Full Stack Fest 2015
• 30mRails Performance monitoring: A primer for developersRishi Jain · RubyConf India 2023
• 26mRails Performance Monitoring 101: A Primer for DevelopersRishi Jain · RubyConf TH 2023
• 44mPanel: Performance... performanceRichard Schneeman, Nate Berkopec, Rafael Mendonça França, Eileen M Uchitelle, Sam Saffron · RailsConf 2017
• 45mRails Performance Monitoring 101: A Primer for Junior DevelopersRishi Jain · RailsConf 2023
• 01h 03mThe performance loop—A practical guide to profiling and benchmarkingDaniel Marbach · NDC Porto 2024
• 01h 00mThe Basics of ProfilingMathieu Ropert · CppCon 2021