In-memory database with indices from scratch
summary
Shows a practical C++ implementation that layers co_await-based fetch and a tiny scheduler on top of libcurl multi API to run many downloads in one thread, exposes utilities like all/optional/all_ranges and cancellation, and uses streamed 3-character n-grams to fetch index shards and intersect postings to answer searches. Demonstrates it hitting about 400–500 MB/s, includes a working hello world demo and reports using it to download and merge large JSON indexes in production.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 3:44High-level async in C++
- 9:04Coroutines in C++
- 15:56Coroutines internals
- 28:43Divide function example
- 30:09Promise type and async ops
- 34:05Scheduling with libcurl
- 37:40Simplified scheduler
- 39:27Multi API for downloads
- 41:16Implementing fetch
- 42:01All and optional promises
- 44:09Ranges and pipable ops
- 45:39Fetching and scraping
- 47:04Search engine with indices
- 49:20Q&A
related talks
16mScaleDB: A Scalable, Asynchronous In-Memory Database
28mThe Memory Efficient Indexing for Vector Databases
01h 05m“A modern database interface for C++"
23mFANTASTIC DATABASES AND WHERE TO FIND THEM
38mAn Ode to 17 Databases in 33 Minutes
50mExploring the benefits of learned indexes in PostgreSQL: Theory and Implementation
46mEdge Performance with In memory NoSQL
27mC++ Search for Database Kernels: Built In, Not Bolted On