Data Domain Cloud Tier: Backup here, backup there, deduplicated everywhere!
Talkat USENIX ATC '19 · on Thursday, 2019-07-11· at 16:10· USENIX ATC Track II: Grand Ballroom VII–IX·
summary
Describes how Dell EMC’s Data Domain appliance implements a cloud tier that preserves deduplication across on-premises and cloud object storage. Covers algorithms and architectural changes for selecting files to migrate (estimating freed on-prem space), moving and restoring deduplicated data across high-latency/cloud-object backends, and deleting expired cloud-resident data while managing dedupe metadata and cost. Includes deployment analysis showing large-scale savings (examples: >20PB logical data and ~40× total compression).
AI-written summary · about
chapters (18)
- 0:00Intro
- 0:33Data Domain Appliance
- 1:22Data Domain Cloud Tier
- 2:22File Representation in Data Domain Filesystem
- 2:55Active Tier and Cloud Tier Architecture
- 4:37Challenges in Deduplication System
- 6:02Requirements for Cloud Tier
- 7:09Physical Scanning
- 8:33How much space will be freed if we move data to cloud tier?
- 9:38How can we seed data efficiently to the cloud tier?
- 10:43How can we move incremental data to cloud tier?
- 11:34How can we efficiently clean dead data on cloud tier?
- 12:30Evaluation Methodology
- 13:09Field Analysis
- 13:32Summary of Field Evaluation Results
- 14:08Internal Evaluation !
- 15:36Conclusion
- 20:24THE ADVANCED COMPUTING SYSTEMS ASSOCIATION
related talks
21mEnsure AWS data recoverability, ft. MassMutual & Boston Scientific
17mMastering the Hybrid Environment: Essential Strategies for DBAs
20mTiDedup: A New Distributed Deduplication Architecture for Ceph4
28mQuantifying the Business Value of Cloud-Native Data Management
12mRevolutionizing Data Management
24mData Masking & Virtualization
37mExtending your Data Storage to the AWS Cloud
11mBreaking Data Silos: Enterprise Data Management