Facebook's Tectonic Filesystem: Efficiency from Exascale
summary
Claims a single exabyte-scale single cluster replaces many special-purpose systems, storing trillions of objects and consolidating an entire data center. Architecture uses a client library for client-side orchestration that batches appends, does in-memory Reed‑Solomon (RS) encoding (default 8 MB chunks, RS 10 → 80 MB encoded blocks), and uses triple-replicated partial-block appends for low-latency small blobs. Production numbers show multi-tenant clusters serve peaks by using stranded IOPS from other workloads, delivering simpler operations and measurably better resource utilization.
AI-written summary · about
chapters (8)
- 0:00Intro
- 0:42Exabyte-Scale Storage Use Cases at FB
- 1:52Storage Infrastructure Before Tectonic
- 3:31Tectonic Overview
- 4:40Scalability: Support Exabyte Scale Clusters
- 7:37Performance: Match Specialized Systems
- 8:06Tenant-specific Optimizations: Appends
- 11:17Tectonic Provides Datacenter-Scale Storage
related talks
19mConsolidating Storage Backends at Scale with Tectonic Filesystem
17mSingularFS: A Billion-Scale Distributed File System Using a Single Metadata Server
01h 01mPanel: Facebook's Distributed Database Evolution
47mWeb-Scale IT Engineering.
21mBuilding Distributed Platforms at Extreme Scale on Fast Changing Data
16mHadaFS: A File System Bridging the Local and Shared Burst Buffer for Exascale
20mscoutfs: Large Scale POSIX Archiving
22mEveryone Loves File: File Storage Service (FSS) in Oracle Cloud Infrastructure