Managing Postgres at scale: Challenges, Tools & Techniques
summary
Defines "at scale" as tens of terabytes or hundreds-to-thousands of databases and lists concrete fixes: use partitioning to drop a year partition instead of slow DELETEs; run WAL archiving and alert on failures; use pgBackRest for incremental/full backups; surface query hotspots with pg_stat_statements and ship metrics to Prometheus/Grafana; deploy Postgres operators on Kubernetes to automate HA, backups and upgrades; use PgBouncer for connection pooling; tune autovacuum workers and thresholds to prevent bloat.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:24Defining “At Scale”
- 3:08Challenges of Managing Databases at Scale
- 5:46Database Architecture & Design
- 16:30Database Monitoring & Alerting
- 19:33High Availability & Disaster Recover
- 22:42Optimising for Hybrid Workloads
- 27:37Maintenance Tasks
- 29:05Summary
related talks
36mPostgres at any Scale
37mPostgres at Any Scale
48mHow to Tame A Mastodon: Lessons for PostgreSQL at Scale
58mScaling Postgres
46mPostgres Platform "Best Practices" for the Modern DBA
51mScaling Postgres to the next level at OpenAI
01h 41mPostgres Database Performance
18mPower Hour, Managing Databases at Scale – Peter Zaitsev