How Atlassian is migrating 40PB data with 99.99% reliability
summary
Atlassian migrates 40+ PB of transactional media into a new transactional data platform using a three-phase migration: dual-write validation, cutover of new writes and backfill, then final read-cutover. Files copy with checksum verification plus a second verification pass; deletions are soft for 30 days. Pre-signed URLs let the target pull from S3, cutting ALB hops, lowering backfill cost ~20% and speeding transfers up to 5x. CDN reduces cross-region latency (~25 ms). Monitoring tracks 50+ metrics across 15 regions; result: 99.99% reliability, zero downtime and zero data loss after onboarding 45 services.
AI-written summary · about
chapters (11)
- 0:00Atlassian’s 40PB migration journey
- 2:07Transactional data migration challenges
- 5:15Transactional Data Platform overview
- 8:50Need for unified migration strategy
- 10:14Phased 40PB migration approach
- 12:01Zero-downtime 40PB move
- 14:55No data loss during migration
- 16:46Cost reduction with pre-signed URLs
- 18:29Cross-region latency handling
- 19:55Migration monitoring at scale
- 20:37Migration results and learnings
related talks
22mFireside Chat with AWS
08mHow to evaluate your data migration and transfer use cases
35mMigrating Your Old Server Products to Be Stateless Cloud Services
27mAccelerate and Simplify Your Data Migration Journey to AWS
16mReal-Time Solution for Critical Data Migrations and Integrations
28mCitus Cluster: our zero-downtime migration story
31mZero Downtime Migrations at Scale
27mModernize Your Data Warehouse