Getting from Schemaless Ingest to Fast SQL
summary
Rockset delivers low-latency SQL on schemaless JSON by ingesting data in seconds and answering selective queries in milliseconds. It implements a C++ IValue tagged-union plus multiple concrete vectorized columnar batches (ints, dict-encoded strings, timestamps) and uses template-driven compilation (generic C++ closures) to hoist type checks out of inner loops. Nested arrays get an inverted index with wildcarded array paths so array-contains becomes a single point lookup — production system, working and efficient.
AI-written summary · about
chapters (5)
- 0:00Real-time analytics intro
- 3:57Schemaless ingest challenges
- 8:13Vectorized query engine
- 14:30Efficient nested indexing
- 18:21Summary
related talks
20mEnabling Machine Learning through Real-Time Data Processing
31mBe a Productive Dev with Real time Analytics as an API
22mHow We Built SQL Rollups on Streaming Data
37mMoving on from RocksDB to something FASTER
30mSurrealDB: The Database Revolution ✨ NewSQL ✨
39mHow we isolate streaming ingest from search using RocksDB
33mSchemaless SQL The Best of Both Worlds
29mAccelerating Performance: Mastering RocksDB for High Speed Data Processing