A Deep Dive into the DB Connection Pool
summary
Mattermost moves from one database per customer to multi-schema sharing via Postgres schemas and PG bouncer but hits a hotspot: three per-customer pools (each size 20) create ~60 client pools contending for 20 outgoing connections, causing frequent connect/disconnect and CPU spikes. A custom proxy Perseus (github com/mattermost/perseus) is built that stores a client schema, sets search_path on checkout, and implements a pool with an idle connection slice, request queue, and a connection reaper; it proves the idea but is discarded for a security risk.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 1:26Mattermost cloud scalability
- 3:12Multi-tenancy with PostgreSQL
- 5:38Connection pooling problems
- 7:48Introducing Persus pool
- 9:14How pools work internally
- 11:24Deployment and security
- 12:48Key takeaways and conclusion
related talks
20mElephant by the pool: PostgreSQL connection poolers overview
27mThe world's worst pool party: Connection management with Prisma
46mPgBouncer Connection Pooling and Routing
18mBest Practices for Connection Management and Memory Allocation
51mConnection pooling routing and queueing with pgBouncer
13mBuilding reliable infrastructure with Postgres
55mProxySQL Use Case Scenarios
59mComparing Connection Poolers for PostgreSQL