Ceph RGW Bucket Snapshots: Beyond Versioning
summary
Ceph RGW adds bucket-level snapshots by extending S3 versioning with a monotonic snap ID dimension so every object instance maps to a snapshot and listings can return a consistent, immediate point-in-time view. Snapshots are created instantly (no full scan); deletion runs a lifecycle sweep to remove unreferenced instances. API adds snap_id and snap_range for retrieval and diffing. Implementation uses bucket index and modified object-heads, is zone-local today, and enforces a configurable few-hundred default snapshot limit while still being development-stage.
AI-written summary · about
chapters (21)
- 0:00Introduction and motivation
- 1:10Why bucket snapshots instead of versioning
- 3:20Capabilities of bucket snapshots
- 5:06Implementation with S3 API
- 7:19How it extends versioning internally
- 9:27Handling non-versioned objects
- 10:52Modifying the logical head (OH)
- 11:55Bucket index and tracking changes
- 13:44Complications in snapshot handling
- 14:47Snapshot removal and cleanup
- 16:12Versioned buckets and snapshots
- 17:41Limitations and considerations
- 18:22Zone-local feature and scalability
- 19:50API extensions and parameters
- 21:16Live demo introduction
- 25:25Demo walkthrough and results
- 25:46Q&A: Locking during snapshots
- 26:50Q&A: Sharding and metadata impact
- 29:00Q&A: Snapshot creation speed
- 30:22Q&A: Metadata and PG concerns
- 31:28Summary and closing
related talks
29mCeph Object Storage Overview, Capabilities and Future Plans
46mMastering Snapshots in OpenSearch: From Basics To Advanced Strategi...
48mMastering OpenSearch Snapshots: From Basics To Advanced Strategies
33mConsistent Volume Group Snapshots, Unraveling the Magic
28mCSI Volume Snapshots: On the Way to Faster and Better Backups
30mKubernetes Data Protection WG Deep Dive
30mKubernetes Data Protection WG Intro & Deep Dive
36mYour Manila CephFS Share Backups Belong to S3