Migrations - The Hardest Actual Problem in Computer Science
summary
Explains a two-year DoorDash migration from a Django/Postgres monolith to services using gRPC, backfill syncs and dual-write comparisons; an interim shared-database approach ships a new client faster but creates cache and operational complexity and occasional data corruption; AWS Aurora replication limits force some manual upgrades; requirements force strict forward/backward compatibility and rolling rollbacks; concrete numbers and costs appear — e.g. 25 engineers for a year on a similar monolith — and the work often doubles runtime costs.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 2:56Why Are Migrations Hard?
- 7:48DoorDash Migration Example
- 14:49Making Changes to Always-On Systems
- 18:50This Sounds Bad
- 24:10Database "Dual Write"
- 26:00Surprising Third Option
- 27:28What Did DD Do?
- 35:04Ways to Make Migrations Easier
- 38:41Good Abstractions
- 43:48Why Is It REALLY So Hard?
related talks
24mMigrations under Production Load: How to Switch...
36mTales from the Postgres Front, and What We Can Learn
33mPeaceful Django Migrations with Efe Öge
31mHelp! My Database Is The Bottleneck
31mSpace Django: Migrating and Redesigning a Database while Hunting for Satellites.
30mWhy most technology migrations fail (and what we've learned along the way)
33mAdding zero-downtime migrations strategy in SaaS project
47mMigrating to Microservice Databases