Journey from Aurora Postgres
summary
DoorDash migrates from single-writer Aurora Postgres to CockroachDB by building a reusable extraction suite that tails table changes, checkpoints and suspends/resumes work, and autotunes parallel range-based imports. The pipeline uses foreign data wrappers when possible, patches pgbouncer for hard cutovers, records deletes to an outbox, and performs schema transforms (sequential ID → UUID, prefixing keys). The team extracts 54 tables to seven clusters, cutting main-cluster load and reducing QPS from 1.6M to 100k; cutovers are lossless and reversible.
AI-written summary · about
chapters (16)
- 0:00Welcome/Introduction
- 2:01DoorDash's Django monolith / single database cluster
- 2:18Implementing microservices into the monolith
- 3:49Survival strategies at DoorDash
- 4:24When DoorDash went down on April 17, 2020
- 8:07DoorDash's extraction tooling
- 10:03Why you should invest in reusable tooling
- 10:55Traditional migration pitfalls
- 13:11The importance of keeping it simple!
- 14:51Customizing runtime behavior
- 16:09Schema transformations
- 18:22Runtime behavior (continued)
- 19:48Table extraction to Aurora Postgres (before and after cutover)
- 20:27Forklift extraction to CockroachDB
- 21:33aurora-extract in Slack
- 22:22CockroachDB migration demo
related talks
15mJourney from PostgreSQL to CockroachDB: Scaling Cisco AI Defense
23mWhy Squarespace adopted CockroachDB for better global performance at scale
24mNavigating Distributed Databases: Challenges & Opportunities
14mBuilding CockroachDB for performance at scale
27mBeyond the Roadmap and Into the Future of CockroachDB
49mPostgreSQL’s Journey to Distributed Databases
17mMigrating Primer’s Live Payments Platform to CockroachDB with Zero Downtime
52mManaging PostgreSQL changes for the Agile DBA