A Cloud-native Architecture for Replicated Data Services
summary
Proposes a main-delta architecture that keeps a single cloud-hosted read-only main and local per-node deltas, prototypes changes in Kafka and Cassandra (md-kafka and modified compaction), and maps memtables to deltas and SSTables to main. Evaluated on AWS EBS/EFS: Kafka shows 3x storage savings and md-kafka matches original throughput on EBS and ~2x better on EFS; Cassandra shows ~2.8x storage savings with similar read/write throughput, proving the approach works with minor changes.
AI-written summary · about
chapters (14)
- 0:00Intro
- 0:22Outline
- 0:48Problem setting
- 1:12Problem overview
- 2:03Typical replicated application on-premise
- 2:38Typical replicated application on Cloud
- 4:43Contributions of this work
- 5:25Quick recap of main-delta architecture
- 6:04Solution overview
- 6:45Merging Deltas to Main
- 8:28Case-study : Delta merge for single primary
- 9:36Case-study 2: Delta merge for quorum system
- 11:26Evaluation
- 13:17Conclusion
related talks
47mPatterns for Persistence and Streaming in Cloud Architectures
38mRe-architecting Apache Kafka for the Cloud
31mZero to data infrastructure
57mMulti Tenant NoSQL and NewSQL Cloud data Design Patterns Deep dive
01h 03mReinvente su estrategia de datos con Lagos de Datos Transaccionales
38mGetting Started with Delta Lake
16mDEPART: Replica Decoupling for Distributed Key-Value Storage
43mWhy We Built Our Own Distributed Column Store