Optimizing Data Storage Costs in Object Storage
summary
Describes Dispersed Object Store (DOS), an object storage developed inside the Mailion corporate mail system, and how deduplication of whole emails and their fragments at the object-storage layer reduces disk space and I/O for corporate mail workloads. Covers the problem of many partially or fully duplicate messages in corporate correspondence and the storage-level deduplication approach used to cut storage costs.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 2:53Storage resource balance
- 4:40Email data properties
- 6:03Data compression settings
- 7:50Object deduplication efficiency
- 9:57Segment deduplication and caching
- 12:47Intuitive vs experimental approach
- 15:18Objective function optimization
- 17:44RBFOpt optimization tool
- 19:09Environment and data
- 21:16HDD cost objective
- 23:20All disks cost objective
- 24:04CPU accounting objective
- 25:07I/O operations objective
- 29:45RAM accounting objective
- 31:09Optimization results
- 32:58Q&A
related talks
14mLeveraging software optimization to reduce cloud footprint
50mMinIO: Scaling and Operations
26mBig Data Is Modern Data
18mMedFS: Pursuing Low Update Overhead via Metadata-Enabled Delta Compression for...
37mDerived Source: Slash Storage Costs Without Losing Data in Open...
25mJourney To Cost-Optimized OpenSearch: A Deep Dive Into Segment Replication
51mModern Email
25mOptimizing Container Registry Storage: Introducing Blob De-Duplication for Accurate Sizing and Cost