Realtime Warehouse Analytics at Picnic with TimescaleDB
summary
Picnic builds a realtime analytics platform on TimescaleDB extension to Postgres that ingests WebMQ events via a collector (optional REST enrichments), stores high‑cardinality time‑series and serves dashboards through Grafana SQL and an Angular operational UI. Requirements include 24‑hour near‑time visibility for ~5,000 shoppers, ~10,000 products, ~40,000 totes and >1M picked items/day; a 2.7M‑record demo runs as hyper tables with continuous aggregates plus 2.7M-record demo gap‑fill, compression and retention policies, and the setup proves performant and highly available.
AI-written summary · about
chapters (12)
- 0:00Introduction and background
- 3:33Real-time warehouse analytics goals
- 9:34Requirements for operational data
- 13:26Previous attempts and challenges
- 19:11Evaluating time series databases
- 23:28Comparing QuestDB and TimescaleDB
- 31:32Final decision and key arguments
- 36:12Picnic’s TimescaleDB setup
- 39:42Demo overview and data model
- 40:44Creating hypertable and querying
- 47:09Hyper functions and continuous aggregates
- 48:38Data retention and cleanup
related talks
47mBuilding a Real-Time Analytics Database
13mTransform your business with time-series data (PRT061)
34mReal-Time Analytics: Going Beyond Stream Processing with Apache Pinot
51mPostgreSQL, The Time-Series Database You Want
49mIntroduction to Real-Time Analytics with Apache Pinot
39mTimescaleDB: Re-engineering PostgreSQL as a time-series database
32mDiscover the Modern Way to Build Real-Time Analytics Solutions at Any Scale
51mBuilding a scalable time-series database on PostgreSQL