Autoresearching Ruby Performance with LLMs
summary
AutoResearch runs LLM agents against benchmarks and opens PRs using Pi auto research and simple tools; one Sidekiq run reports 21,000 jobs/sec and yields micro-optimizations — replacing a 3-line global lock with a striped-lock expansion and swapping time.now.to_i for process.clock_gettime (saving ~20 ns) that together give about a 1% speedup. Some projects (Liquid, Ruby LSP) see meaningful gains, but many AI PRs are unmergeable due to complexity, caching bugs, weak tests, or mismatched assumptions.
AI-written summary · about
chapters (12)
- 0:00Introduction and audience participation
- 1:53What is auto research?
- 3:57Auto research in real PRs
- 6:25Introducing Pi coding agent
- 8:12Sidekiq auto research demo
- 10:44Why generated code isn't merged
- 13:56The architect vs the pirate
- 15:44The bitter lesson in AI
- 17:31Loops and gates in AI workflows
- 19:15The software factory concept
- 21:46Open questions about AI factories
- 24:56Conclusion and takeaways
