Migrating from Monolithic to Cloud Native Distributed SQL Database with Zero Downtime
summary
Tokopedia moves from monolithic Postgres to YugabyteDB with no user downtime by doing schema and query rework, provisioning a multi-node universe (example: 10 nodes, replication factor 3), running CDC via Kafka (wal2json snapshots → Kafka → custom Sarama consumer into YugabyteDB) to avoid double writes, splitting traffic (5/5) while validating data integrity, load-testing to ~100k rps, and using a feature toggle to flip connections; Airflow handles batch exports to Bigtable.
AI-written summary · about
chapters (8)
- 0:00Monolith to distributed SQL
- 2:30Gojekopedia's monolith challenges
- 5:41Pre-migration checklist
- 9:15CDC pipeline setup
- 13:31Zero-downtime migration
- 15:37Data pipeline integration
- 17:21DBAs and dev runbooks
- 20:13Training distributed SQL
related talks
26mLeveraging a Distributed SQL Database at Tokopedia: Best Practices & Lessons Learned
35mScaling UPI for the Future: Zero Downtime Migration to
28mDisrupting the Status Quo with Distributed SQL to Deliver Data-Driven Solutions
26mIntroducing YugabyteDB for PostgreSQL Users
13mStrategic Shift to YugabyteDB as Core Database
20mDatabase migration: Insights on Dual Read Write Strategy
36m5 Hard-Learned Lessons from Successful Database Migration and Modernization
51mEmbrace, Extend, Evolve - Making PostgreSQL Work for a Cloud Native World