PostgreSQL queues done right with PgQ
summary
Postgres queue PgQ claims to replace SKIP LOCKED designs with a lockless, MVCC-based approach that keeps throughput high and bloat low. It stores events in three rotating child tables, uses a single txid index per child, and defines batches by ticks every second so consumers read transaction-consistent snapshots. A small daemon pgqd (or a pg_cron wrapper) handles retries every 30s and maintenance every 120s, producing zero dead tuples and minimal write overhead.
AI-written summary · about
chapters (14)
- 0:00Music & introduction
- 0:26Agenda: queues, problems, PgQ
- 1:15Why use in-database queues
- 2:16Naive queue table design
- 3:05Consuming with skip locked
- 4:03Scaling issues & table bloat
- 6:02Visibility timeout workaround
- 9:11Partitioning and pointer approach
- 11:08PgQ architecture & core concepts
- 13:25PgQ API: queues, batches, consumers
- 16:49Internals: ticks, batches, MVCC
- 20:00Running PgQ on managed cloud
- 23:00Replacing pgqd with pg_cron
- 26:53Final recap: why PgQ wins
related talks
30mQueues in PostgreSQL
23mRethinking Message Queues: PostgreSQL's Role
48mUsing Asynchronous Message Queues with PostgreSQL
43mBlazingly Fast Message Queue on Postgres with Rust
23mPostgres as a Message Broker
44mBuilding a Distributed Queue with Delayed Jobs on PostgreSQL
26mSkip the Queue! Event-Driven Apps for Everyone with Postgres!
31mBatch Job / Queue Processing Without a Message Broker