PostgreSQL as a distributed computing platform

on YouTube ↗38m

" summary

Argues PostgreSQL can act as a distributed platform and shows a working prototype: a 4-node Citus cluster with producers writing to a distributed events table, consumers claiming leases per shard via a probabilistic algorithm, and consumption driven by logical decoding via replication slots using DB link RPC. Key tools: Citus, dblink, pg_cron, logical decoding functions; core logic is two PL/pgSQL routines (obtain_leases and pull_events) of about seven SQL statements each, total prototype ≈300 lines. System achieves scaled consumers (32 vs 16 leases) and transactional exactly‑once delivery.

AI-written summary · about

" chapters (9)

1. 0:00Start
2. 0:50Small data architecture
3. 7:12Distributed system in progress
4. 13:49Sharding strategy
5. 15:05Consumer leases
6. 20:11Consumers
7. 22:18Consuming events
8. 25:59Consumer loop
9. 26:40Failure handling

" related talks

• 08mFoundations of Distributed PostgreSQL with CitusOnder Kalaci · Azure Cosmos DB: Liftoff 2022
• 31mKEYNOTE: The Distributed PostgreSQL Problem & How Citus Solves itMarco Slot · Citus Con: An Event for Postgres 2023
• 32m8 keys to the growth of Citus & Distributed PostgreSQLUmur Cubukcu · Citus Con: An Event for Postgres 2022
• 44mDistributed Postgres with CitusWill Leinweber · HighLoad++ 2017
• 53mBreakout SessionJason Petersen · PGConf Silicon Valley 2015
• 55mBreakout SessionDan Robinson · PGConf Silicon Valley 2015
• 42mPostgreSQL Distributed: Architectures & Best practicesMarco Slot · PostgreSQL Conference Europe 2023
• 46mpg paxos Table Replication through Distributed ConsensusMarco Slot · PGConf Silicon Valley 2016