Could Ceph Become More Like a Database?
summary
Proposes adding RADOS Native Indexing primitives so an object write can carry a tag that updates its index atomically at the OSD in a single RADOS transaction, removing the current client-side three‑stage commit for OMAP indexes; requires collocating data and index entries (use of object locator/hash overrides) and blocking list operations until recovery, and needs batching and new resharding care because welded metadata inflates data movement. Notes EC pools gain OMAP soon and Eric’s ordered bucket index both helps and complicates this.
AI-written summary · about
chapters (7)
- 0:00Introduction and motivation
- 7:52History of OMAP and key-value storage
- 12:00Limitations of current OMAP design
- 19:25Native indexing concept proposal
- 27:17Ordered bucket index and collocation
- 34:19Challenges and trade-offs of native indexing
- 44:48Discussion and future possibilities
related talks
29mCeph Object Storage Overview, Capabilities and Future Plans
33mThe Challenge of Storing Small Objects on a Large Scale
50m#MySQL on Ceph using Rados
43mCeph and Flash
38mLight and Adaptive Indexing for Immutable Databases
51mSix New Important RocksDB Features And Planned Works
20mBetter Indexing and Faster Retrievals for Decentralized Hot Storage
01h 13mC++ Memory Model Meets High-Update-Rate Data Structures