Large Scale NoSQL DB Migration Under Fire
summary
Migrates 1.8 billion records from Couchbase into Aerospike across 43 nodes with zero downtime by changing the application to dual-write into both databases, then switching reads to Aerospike while falling back on misses. Uses a hit‑miss metric to know when migration is complete, replays events via Kafka as a buffered loader for missing keys, practices cluster resizing, and reports about a one third reduction in infrastructure cost after roughly two months of staged rollout.
AI-written summary · about
chapters (11)
- 0:00Intro
- 1:18Agenda
- 5:11Couchbase
- 7:25Language
- 8:40Methodology
- 10:54Testing
- 14:50Resize
- 19:52Leiby API
- 21:11Message Queue
- 23:21Summary
- 25:48Data Transfer
related talks
23mMigrating from HBase to Aerospike for Edge and System of Record Application
49mJourney with Aerospike: Architecture Best Practices
31mUser Attribution at Scale: All-Flash, ½ Trillion Objects
48mRebuilding a RT Datastore: the design, deployment & performance story
24mWhat a Modern Database Enables
36mHot Swap Your Datastore: A Practical Approach and Lessons Learned
06mSnapdeal and Aerospike
29mWhat breaks when Aerospike hits 6 million QPS