Easy rewrites with ruby and science!
summary
GitHub uses the open-source dat-science gem and companion analysis gem to run side-by-side experiments while rewriting the permissions model. The team dark-ships the new abilities table, runs dat-science gem experiments measuring mismatches and timing, finds about 20 mismatches/hour and many data-quality bugs, builds tools like throttl, and iterates until experiments are green. A performance change removes 72 million rows and the work is dark-shipped experiments monitored in graphite before the new system goes live mid‑August.
AI-written summary · about
chapters (5)
- 0:00Start
- 9:10tests weren't modeling production data
- 12:57lessons learned
- 22:26performance problems
- 25:21One Last Data Quality Issue
