Apache Ozone - State of the Union
summary
Apache Ozone is a Raft-based scalable object store with decoupled namespace and block management, using RocksDB for metadata and an S3 gateway plus an OFS Hadoop-compatible filesystem. It claims scales to a billion files (about 10× HDFS), uses Ratis-based consensus for OM/SCM replication and SCM-HA, and introduces a prefix (FSO) layout with ~5% write amplification that yields 10–15x faster drop-table (12s → <1s) for Hive-style renames/deletes. Roadmap highlights erasure coding, container balancer, streaming pipeline and tenant-aware S3.
AI-written summary · about
chapters (14)
- 0:00Introduction and speakers
- 1:11History and overview of Ozone
- 3:19Ozone architecture and components
- 6:46Deployment architecture of Ozone
- 10:40Metadata layout in Ozone
- 13:07Features in Ozone 1.1.0 release
- 17:27Ozone community and contributors
- 18:31New features in Ozone 1.2.0
- 18:52Ozone File System optimizations
- 25:09Storage Container Manager HA
- 30:29Non-rolling upgrades in Ozone
- 33:58Ozone Manager preparation for upgrades
- 36:04Ozone roadmap and upcoming features
- 38:49Q&A session
related talks
32mElevating Scalable Object Storage: A Comprehensive Exploration of Ozone’s Trailblazing Capabilities
24mOxia - A Horizontally Scalable Alternative to Apache Zookeeper
21mApache Ratis - A High Performance Raft Library
50mA Homegrown S3 Server: Challenges of Building Stable Storage
29mEnhancing Flexibility and Productivity with Access Patterns and Storage-Agnostic Abstractions
38mRe-architecting Apache Kafka for the Cloud
33mH2O internals
25mCloud native data replication strategies