Rethinking Message Queues: PostgreSQL's Role
summary
Argues that PostgreSQL can replace a separate broker for many use cases by implementing a jobs table queue and a push channel, showing LISTEN/NOTIFY for low‑latency non‑durable pushes and a pull pattern using FOR UPDATE ... SKIP LOCKED that raises concurrent throughput by orders of magnitude in demos. Demonstrates retries, dead‑letter tables and priority columns, warns of table bloat, index decay and long transactions, recommends vacuum tuning or partitioning/truncation for workloads above thousand jobs per minute.
AI-written summary · about
chapters (10)
- 0:00Introduction and speaker intro
- 0:44What is a message queue
- 2:46Evolution of message queues
- 4:11Why PostgreSQL fits message queues
- 5:12Push vs pull message consumption
- 9:01Handling concurrent message processing
- 11:05Real-world challenges with PostgreSQL queues
- 13:57Implementing queuing features in PostgreSQL
- 15:22When to choose PostgreSQL over other queues
- 18:30Q&A session
related talks
30mQueues in PostgreSQL
48mUsing Asynchronous Message Queues with PostgreSQL
23mPostgres as a Message Broker
29mPostgreSQL queues done right with PgQ
31mBatch Job / Queue Processing Without a Message Broker
44mBuilding a Distributed Queue with Delayed Jobs on PostgreSQL
44mPushy Postgres and Python
01h 05mDatabases: Bespoke Tailoring and Perfect Fit