A New Data Model for Time Series Data Processing
Talkat Open Source Summit Europe 2023 · on Wednesday, 2023-09-20· at 12:20· Room 0D-2-0D-3 (Floor 0), Abandoibarra Etorb., 4, 48011 Bilbao, Bizkaia, Spain·
summary
Proposes one table per sensor data model with columnar, block-oriented storage that keeps timestamps ordered, stores tags separately, and precomputes block aggregates; introduces supertable templates to manage millions to 1 billion tables. Claims data ingestion is 2–5× faster, query latency is 2–30× lower and compression is often much higher versus Prometheus/InfluxDB/TimescaleDB, with TSBS benchmark scripts provided for reproduction. Architecture uses sharded vnodes, three-way replication via Raft, and separates compute from storage for fast restarts.
AI-written summary · about
chapters (11)
- 0:00TDengine and time series data
- 2:33Challenges with existing time series databases
- 5:18Time series data traits
- 9:59Relational model limits
- 13:15One-table-per-device model
- 18:31Aggregation with super tables
- 25:23TDengine vs competitors
- 27:27Scalability and distributed
- 30:41DB with queue and cache
- 31:44Open source growth
- 33:53Q&A
related talks
39mA New Data Model for Time Series Data Processing
07mTime Series Databases
49mThinking in time series, the new solution to your pro...
29mCatching Waves with Time-Series Data
30mCatching Waves with Time-Series Data
39mTimescaleDB: Re-engineering PostgreSQL as a time-series database
58mIntroduction to Timeseries Databases
51mBuilding a scalable time-series database on PostgreSQL