Data @Scale Seattle- Brian Beach
summary
Backblaze builds a horizontally scalable storage system where clients upload directly to rackmount pods (45 drives, ext4) and a vault is 20-node vault with 1-drive-per-node tomes that stripes data and parity across pods; design targets are 2.5 GB/s ingest and supporting ~1 MB average object size. Implementation uses Java Reed‑Solomon encoding (open sourced) running on 10 GbE NICs with ~24 GB RAM for filesystem cache, rebuilt drives in about two days, and five vaults already hold ~18 PB while the company stores ~150 PB total.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 0:52What is Backport
- 2:40Backblaze Storage Pod
- 9:31Project Goals
- 11:13Architecture Changes
- 12:34Design Decisions
- 15:09Storage
- 16:38Housekeeping
- 20:02Scaling
related talks
31mScaling Autonomous-Driving R&D: A 100 PB, 100 Billion-File Open-Source Elastic File...
28mThe Ultimate DataHub for all your BigData needs
54mPrimary storage for SaaS solutions
29mAccelerate your cloud architectures with the right storage options
24mCloud Workloads At Petabyte Scale
21mBuilding Distributed Platforms at Extreme Scale on Fast Changing Data
15mAWS On Air Voice of the Customer ft. Aerospike, Inc.
23mArchitecting for Performance, Scalability and Real-time Data with AWS