Breakout Session

on YouTube ↗55m

" summary

Claims that PostgreSQL + CitusDB power interactive analysis over ~50 terabytes of event data and tens of billions of rows (transcript cites ~80B events), returning results in seconds and near real time. Architecture uses CitusDB sharding by customer+user, Kafka as a durable commit log with consumer-managed offsets and replays, and UDFs in C and PL/pgSQL (e.g. funnel UDFs) plus partial indexes per event definition to make ultra-sparse retroactive event queries fast.

AI-written summary · about

" chapters (10)

1. 0:00Intro
2. 0:32Quick Overview
3. 7:47Challenges
4. 8:41Data Scale
5. 15:02Distributing With CitusDB
6. 19:54Partial Index Strategy
7. 25:42UDFs For Advanced Analysis
8. 26:55Our PostgreSQL Wishlist
9. 29:03Building A Real System
10. 45:03Future Work

" related talks

• 38mPostgreSQL as a distributed computing platformMarco Slot · PostgresOpen Silicon Valley 2017
• 32m8 keys to the growth of Citus & Distributed PostgreSQLUmur Cubukcu · Citus Con: An Event for Postgres 2022
• 31mKEYNOTE: The Distributed PostgreSQL Problem & How Citus Solves itMarco Slot · Citus Con: An Event for Postgres 2023
• 24mCitus from the Customer eyesSai Krishna Srirampur · Citus Con: An Event for Postgres 2023
• 30mBuilding geospatial apps on Postgres, PostGIS, & Citus at large scaleNirav Savani, Mahipal Ramidi · Citus Con: An Event for Postgres 2022
• 20mLessons learned scaling our SaaS on Postgres to 8+ billion eventsJonathan Denney · Citus Con: An Event for Postgres 2022
• 08mFoundations of Distributed PostgreSQL with CitusOnder Kalaci · Azure Cosmos DB: Liftoff 2022
• 44mDistributed Postgres with CitusWill Leinweber · HighLoad++ 2017