Zero Downtime Migrations at Scale
summary
Zero downtime migration migrates Drive backups from legacy storage into Spanner at global scale with no visible user impact by combining dual-writes to old and new stores, a large backfill pipeline that copies >1 billion backups (over 1 trillion objects) and stepwise rollouts from 1% upward. Operations use probers comparing bytes and query results, heavy validation across layers, exhaustive observability and capacity planning, migrate backends first (persistent queue replacing scheduler in the build example), and the project succeeds with gradual scale ups and verified parity.
AI-written summary · about
chapters (12)
- 0:00Introduction and overview
- 2:35Types of migrations
- 6:28Zero downtime necessity
- 8:13Migration at scale for Drive backups
- 12:25Data vs architectural migration
- 13:08Writing new data to both systems
- 15:11Backfilling existing data
- 17:00Probing the new stack
- 18:47Hardening the system
- 20:33Scaling the migration
- 23:45Key takeaways for migrations
- 30:04Q&A and closing
related talks
24mZero Downtime Mastering Cassandra Live Migrations at Scale
37mDatabase Schema Migrations with Zero Downtime
19mZero-Downtime by Design: High-Availability Migration Patterns
33mDatabase Schema Migrations with Zero Downtime
23mHow Atlassian is migrating 40PB data with 99.99% reliability
35mDatabase migrations without downtime
34mZero Downtime Database Migrations and Deployments
35mScaling UPI for the Future: Zero Downtime Migration to