Building RailsPerf, a toolkit to detect performance regressions

on YouTube ↗30m

" summary

Practical account of building RailsPerf: a per-commit regression detector that runs per-commit benchmarking against Rails using a suite of microbenchmarks (ActiveRecord, ActionController, router, views), measuring timings and allocations, isolating runs with docker isolation, and posting results back to GitHub. Prototype embeds into ruby-bench, runs on dedicated hardware, has ~10–20 Rails benchmarks, contributes to ~250k builds, and already identifies regressions that get fixed the next day. The repo, CI flow, and visualization are all implemented.

AI-written summary · about

" chapters (16)

1. 0:00Start
2. 0:12Building a toolkit to detect performance regressions in Ruby on Rails core
3. 0:47Catching Performance Regressions in Rails
4. 1:27What's a performance regression?
5. 1:46What's a performance regression in Rails?
6. 2:16Case Study
7. 2:33Discourse
8. 3:53Faster framework = Faster app
9. 4:23Examples from Rails commits
10. 4:35Metrics
11. 5:36Basic primitives
12. 11:54Optimize only "hot" code
13. 12:46The Idea to build a service
14. 16:57Correct benchmarks
15. 19:00The Service
16. 24:28250k builds for Ruby

" related talks

• 26mRails Performance Monitoring 101: A Primer for DevelopersRishi Jain · RubyConf TH 2023
• 37mThe Future of Ruby Performance ToolingAaron Quint · GoRuCo 2014
• 21mBuilding a Performance Analytics Tool with ActiveSupportChristian Bruckmayer · RailsConf 2020 CE
• 30mRails Performance monitoring: A primer for developersRishi Jain · RubyConf India 2023
• 01h 02mPerformanceJeremy Kemper · RailsConf Europe 2008
• 35m"10 performance sins of Rails applications"Aleksander Dąbrowski · RubyC 2019
• 29mUpgrading Rails at ScaleEdouard Chin · RailsConf 2018
• 28mRESCUE SQUAD: Rails Edition!Ben Lovell · Ancient City Ruby 2015