Self Hosted Citus using Cloud Computing

on YouTube ↗25m

" summary

SafetyCulture moves a high‑write SaaS workload onto a self‑hosted Citus cluster to handle a >12 TB hottest table with thousands of TPS; tooling and practice include automated AMI builds with Packer+Ansible, infrastructure as code via Terraform, HA via Patroni, backups with pgBackRest, Prometheus monitoring, and synchronous replication for durability. Data migration uses foreign data wrappers and pglogical where possible, bulk loads use FDW-based insert-for-select, several Citus and kernel networking bugs appear and get patched, and production performance and customer experience improve.

AI-written summary · about

" chapters (10)

1. 0:00Introduction
2. 0:44Is self hosting Citus what you need?
3. 4:21Do you have a business case?
4. 7:31Tooling choices for self-hosting Postgres
5. 10:30Principles for production infrastructure
6. 11:19AMI creation
7. 13:36PostgreSQL HA
8. 19:10Things that caught us out
9. 20:42Testing and validation
10. 22:21Going live

" related talks

• 28mCitus Cluster: our zero-downtime migration storyMatthieu Dumont, Antoine Hemery · Citus Con: An Event for Postgres 2022
• 52mScaling a Saas application beyond a single Postgres with CitusJesse H. Willett · PostgresOpen Silicon Valley 2017
• 32m8 keys to the growth of Citus & Distributed PostgreSQLUmur Cubukcu · Citus Con: An Event for Postgres 2022
• 16mOnline Scale-Out and Cloud Data IngestionAdam Wolk · Azure Cosmos DB: Liftoff 2022
• 32mLoading at Hyperscale: Advanced Data Import Techniques for CitusColton Shepard · Citus Con: An Event for Postgres 2022
• 44mDistributed Postgres with CitusWill Leinweber · HighLoad++ 2017
• 30mBuilding geospatial apps on Postgres, PostGIS, & Citus at large scaleNirav Savani, Mahipal Ramidi · Citus Con: An Event for Postgres 2022
• 13mWhat I learned creating Postgres monitoring metrics for Citus on AzureSena Gungor · Citus Con: An Event for Postgres 2022