Ingesting over 4 million rows a second on a single instance
summary
Claims ingesting over 4 million rows/sec on a single instance using QuestDB and a 16‑CPU benchmark; a live demo on an SSD Mac shows ~3.2M/sec. Implementation details: Java with unsafe direct memory mapping, custom lightweight standard library, C++ vectorized kernels, ILP for fast ingestion, and new partition splits to avoid rewriting large partitions. Engine auto‑tunes write buffers, uses symbol encoding for strings, and isolates CPU threads so writes stay fast under query load; the peak figure holds under controlled benchmark conditions.
AI-written summary · about
chapters (8)
- 0:00Introduction to QuestDB
- 1:06Design philosophy
- 5:42Ingestion benchmarks and demo
- 12:12Benchmarking challenges
- 15:28When to use time-series DBs
- 21:20Optimizing for time-series
- 27:25Resource management and trade-offs
- 35:24Low-level implementation
related talks
30mIngesting A Million Time Series Per Second On A Single Instance
01h 07mQuestDB: Fast Open Source Time Series Database
40mAccelerating QuestDB: Lessons from a 6x Performance Boost
22mIt’s all about the db - a journey to accommodate 100k events per second
41mHigh-Performance Ingestion with Python and Swarm64DB
45mDealing with TBytes of Data in Realtime
40mDuckDB: Harnessing in-process analytics for data science and beyond
26mReimagine Real-time