The 30-Month Migration
summary
A sustained 30 months effort restructures contacts across two databases into a single Postgres schema in four stages: migrate from Neo4j to Active Record using a Ruby switching model and feature flag, convert UUID primary keys to integer IDs with reversible migration helpers, run SQL CTE upserts (about 10 minutes per realtor versus ~1 day in Ruby) to de-duplicate shared contacts, and finally replace the join table with direct foreign keys plus triggers; extensive cloning, CI, QA swarms and spot checks validate each step and it works.
AI-written summary · about
chapters (27)
- 0:00Intro
- 2:23Three Principles
- 6:57Making a Plan
- 7:27Schema Definition
- 8:13Switching Models
- 11:23Testing
- 12:45Executing
- 13:04Finishing the Job
- 14:56A Spike
- 15:52Problem: Foreign Key References
- 17:04Problem: Polymorphic Tables
- 17:47Discovering Constraints
- 18:22Five Big Migrations
- 18:38From Spike to Solution
- 19:29Being Careful
- 22:40The Strategy
- 24:18INSERT with ON CONFLICT
- 28:08What Drove the Change?
- 32:22The Plan: A 12-Step Epic
- 33:18Tracking Progress
- 33:36Auditing Trigger Activity
- 34:47Slow and Steady
- 35:11Keep Looking Ahead
- 35:34Each Stage Was Different!
- 36:12Leverage Your Database
- 37:21Focus: The Two-Edged Sword
- 38:00What Would We Do Differently?
related talks
51mMigrations - The Hardest Actual Problem in Computer Science
30mPuny to Powerful PostgreSQL Rails Apps
37mDenormalizing Rails
31mModernizing a megascale database
29mMigrating Data Quickly and Safely
52mManaging PostgreSQL changes for the Agile DBA
34mBeyond CRUD: the PostgreSQL techniques your Rails app is missing.
33mAdding zero-downtime migrations strategy in SaaS project