Blazingly Fast Message Queue on Postgres with Rust
summary
An open source Postgres extension called PGMQ implements a SQL API and Rust client via PGX, stores messages as JSONB with optional archive tables, and guarantees temporary exactly‑once delivery using a visibility timeout. Benchmarks on a 16 vCPU 32 GB EC2 host with 5 producers and 40 consumers (batch size 10) show ~3–5 ms latencies and about 20k msgs/s for 22 B messages, performance degrading with larger payloads; autovacuum and partitioning (PG Partman) are critical to avoid bloat.
AI-written summary · about
chapters (10)
- 0:00Introduction and motivation
- 1:31Project origin and use cases
- 7:16PG MQ API and design
- 9:06Visibility timeout explained
- 12:00Underlying Postgres operations
- 18:27Benchmarking results and insights
- 26:20Running PG MQ in production
- 29:33Future plans for PG MQ
- 31:25Live demo of PG MQ
- 41:22Q&A session
related talks
23mPostgres as a Message Broker
29mPostgreSQL queues done right with PgQ
23mRethinking Message Queues: PostgreSQL's Role
48mUsing Asynchronous Message Queues with PostgreSQL
46mFearless Extension Development With Rust and PGRX
31mPostgres without SQL: Natural language queries using GPT-3 & Rust
56mLightning Talks
40mPerformance is a feature: accelerating Postgres