Secure Apache Ozone with High Availability
summary
Apache Ozone adds High Availability (SCM HA in 1.0, OM HA in 1.1) and secures inter-service RPCs with mutual TLS for gRPC and delegation tokens for Hadoop RPC, using a distributed CA model: root CA + SubCA running across SCMs so new nodes get signed certs without syncing secret keys. State and leader election use Apache Ratis (Raft); block I/O uses block tokens signed by OM and verified by datanodes. Demo shows certificate serial IDs, S3 access-key/secret issuance, and end-to-end validation working in an HA cluster.
AI-written summary · about
chapters (21)
- 0:00Introduction
- 0:52Agenda
- 1:37What is Ozone
- 3:55Architecture
- 6:53Security
- 9:05Protocols
- 12:06Signing Keys
- 15:13Token Format
- 15:47Block Token
- 16:09Blog Token
- 16:40S3 Security
- 17:45S3 Credential Generation
- 19:01S3 Token Validation
- 19:38High Availability
- 20:36HSM Security
- 21:42Ozone has Accurate App
- 22:18Security Aspects
- 23:10Token Validation
- 24:07Mutual TLS
- 24:57Endpoints
- 25:29Demonstration
related talks
32mElevating Scalable Object Storage: A Comprehensive Exploration of Ozone’s Trailblazing Capabilities
24mOxia - A Horizontally Scalable Alternative to Apache Zookeeper
25mCloud native data replication strategies
21mApache Ratis - A High Performance Raft Library
21mCentralized Authentication with Apache APISIX and Ory
38mProtecting the Data Lake
31mRings of Security for Big Data in Cloud
26mThe amphorae of application delivery & security with Octavia