What a serverless database means for users
summary
Compares ArcticDB, Parquet (single-file and partitioned) and HDF5 on concrete workloads: 2.5 million rows historical load, 500-row daily appends, single-day updates. ArcticDB uses LMDB locally (single-writer lock) but parallelizes reads and matches or beats Parquet/HDF5 for full reads, while Parquet partitioning is fastest for appends but single-file Parquet is disastrously slow for append/update. ArcticDB exposes as_of versioning natively, writes metadata and indexes during updates, and scales to S3 without adding servers.
AI-written summary · about
chapters (12)
- 0:00Serverless databases intro
- 1:27How serverless databases work
- 3:15Time series data model
- 7:04File systems vs serverless DBs
- 10:39Writing data methods
- 13:29Append and update ops
- 16:18Write performance
- 20:09Testing scenarios
- 22:56Read performance
- 25:45Scaling and versioning
- 27:31Limitations
- 29:36Conclusion
related talks
36mArcticDB, the OLAP antidote
42mColumnar Storage: Redefining Data Management for the Modern Era
26mColumnar Storage: Redefining Data Management for the Modern Era
28mAnalyze data easily with duckdb - and the implications on data architectures
40mDuckDB beyond the notebook
26mBuilding Data Lake Using DuckDB
43mKeep calm and dive deep into newest database innovations
40mDuckDB: Harnessing in-process analytics for data science and beyond