PostgreSQL at Pandora
summary
Pandora migrates from Oracle to Postgres, defines five DBMS classes, and runs a music pipeline from Nexus ingestion to read‑only music metadata exports to a radio tier sharded into 16 shards by listener ID. Monitoring uses custom scripts, tailmail, a historical metrics DB and alerts on config change and rate-of-change. Backups move from pg_dump to pg_basebackup with ~98–99% success but WAL/shipping and transaction‑ID wraparound bite. Operations develop cluster (non‑ACID) with per‑node agents plus streaming replication and replication slots to avoid WAL loss. ~1,650 clusters and 335 TB of data.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 0:45PostgreSQL at Pandora
- 8:01Monitoring Postgres
- 12:26Historical Data
- 17:09Backups
- 23:03PostgreSQL Replication
- 29:02Replication High Availability
- 31:15Cluster
- 34:23Cluster attributes
- 38:31Disadvantages
- 40:23Clusters
- 43:42Scaling issues
- 45:20Vacuum issues
- 48:02PostgreSQL Clusters
- 48:58Summary
related talks
29mTales from the Field - Oracle to PostgreSQL migrations
46mBig databases, many servers, data center or cloud - it's all here!
51mWhen Postgres Can't, You Can!
19mPostgreSQL at Amazon
36mPostgres at any Scale
51mScaling Postgres to the next level at OpenAI
38mWhat we built with Open Source
01h 09mReplacing Oracle Features in PostgreSQL: HA, Functions & Extensibility