Light and Adaptive Indexing for Immutable Databases
summary
Claims that immutable object-backed storage plus small on-disk indexes cut latency by avoiding random reads, with compute-local adaptive structures filling gaps. Uses Arrow for columnar execution, a write‑ahead log for durability and object storage like S3 for bulk files. Shows a concrete index: Radix Spline (single‑pass CDF spline plus a 3‑bit radix layer) where ~1k–10k spline points compress millions of rows. Builds idempotent light indexes in background, uses min‑max and bloom filters and adaptive cracking via quickselect on compute nodes.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 0:25What if Masters were free
- 1:59Vision 86
- 5:08Immutability
- 8:54Composition Storage Models
- 13:48The Problem
- 21:15Learned Indexes
- 27:45Adaptive Indexing
- 29:31Instance optimized system
- 30:44Engineering
related talks
28mAerospike's Journey Through Innovations, Sustainability, and Roadmap
42mColumnar Storage: Redefining Data Management for the Modern Era
37mKEYNOTE: Data Architecture Turned Upside Down
42mContemporary Approaches to Data at Scale
20mBetter Indexing and Faster Retrievals for Decentralized Hot Storage
30mSparkless Patterns: The New Data Stack and Modern Data Architecture
26mColumnar Storage: Redefining Data Management for the Modern Era
11mDatabase Evolution: Partitioning and Indexing