Scaling Postgres to the next level at OpenAI
summary
Postgres runs as a single-primary, multi-replica fleet and handles millions of QPS by moving shardable writes off-platform, reducing writer count in application code, and using PgBouncer connection pooling to collapse thousands of client connections into a few backend sessions. Operational changes include hard rate limits, a 30s session/statement timeout, dedicated replicas for high‑priority reads, removing unused indexes to cut WAL amplification, and fixes for a WAL shipping CPU bug; gaps called out: percentile query histograms, schema-change events, index disabling, and adaptive knob tuning.
AI-written summary · about
chapters (12)
- 0:00Introduction and background
- 2:25Scaling challenges
- 7:02Primary load mitigation
- 9:26Query and connection management
- 11:58System resilience strategies
- 16:05Proxy and availability
- 17:54Schema change management
- 20:28Operational results
- 22:28Case study on outages
- 33:28Replica lag and bug fixes
- 37:13Future feature requests
- 46:58Q&A session
related talks
25mScaling Postgres to the next level at OpenAI
36mPostgres at any Scale
13mBuilding reliable infrastructure with Postgres
37mPostgres at Any Scale
58mScaling Postgres
30mManaging Postgres at scale: Challenges, Tools & Techniques
40mPushing past PostgreSQL’s vertical scaling limits - Running Amazon RDS
50mPostgreSQL Scale-Out at inMobi