scoutfs: Large Scale POSIX Archiving
summary
Open source clustered POSIX file system that manages billions of archive entries (example: ~10 billion files) by turning metadata into the authoritative store and keeping only truncated offline extents for contents. It implements global inode indexes stored in the same atomic key‑value engine, uses LSM trees for parallel writes, maintains a preallocated manifest ring (B‑tree) to let nodes read old versions safely, and accepts compaction bandwidth in exchange for parallelism via large 1 MB streaming writes rather than tiny 4 KB updates.
AI-written summary · about
chapters (8)
- 0:00Intro
- 3:29Software interfaces
- 9:39Nodes
- 12:00Questions
- 12:33Workload
- 15:04Clean up
- 18:00Read levels
- 19:17Starting over
related talks
17mSingularFS: A Billion-Scale Distributed File System Using a Single Metadata Server
15mInfiniFS: An Efficient Metadata Service for Large-Scale Distributed Filesystems
12mFacebook's Tectonic Filesystem: Efficiency from Exascale
22mEveryone Loves File: File Storage Service (FSS) in Oracle Cloud Infrastructure
42mApache Ozone - State of the Union
16mHadaFS: A File System Bridging the Local and Shared Burst Buffer for Exascale
28mImplementing SMB Semantics in a Linux Cluster
19mConsolidating Storage Backends at Scale with Tectonic Filesystem