Lessons Learned from benchmarking and profiling distributed PostgreSQL

on YouTube ↗24m

" summary

Used the Citus Benchmark Suite plus YCSB via JDBC to run 100 million-record tests with 100% INSERT (workload A) and 100% SELECT (workload C). INSERT throughput rises with connections until out‑of‑memory errors near 3600; INSERTs are memory‑limited (~31 connections per GB) and spend most time waiting for WAL flushes. SELECTs peak around 800 connections and then degrade as CPU saturates (3200 connections); diagnosis uses iostat, PostgreSQL logs, and perf with a 0.01% monitor sample.

AI-written summary · about

" chapters (7)

1. 0:00Intro
2. 1:44Architectures for Distributed PostreSQL
3. 5:53The PostgreSQL Query Processing Pipeline
4. 8:56The scope of the research
5. 11:09Benchmarking Citus
6. 17:41Identifying bottlenecks
7. 22:14Key Takeaways

" related talks

• 57mPostgreSQL When It's Not Your JobChristophe Pettus · Postgres Open 2012
• 36mBest Practices in PostgreSQL Tuning: Navigating Key Performance Bottlenecks in the CloudAlicja Kucharczyk, Dilip Ramachandran · SQLBits 2024
• 26mTroubleshooting High CPU Utilization in PostgreSQL DatabasesDmitry Romanoff · Citus Con: An Event for Postgres 2023
• 43mTuning PostgreSQL for High Write WorkloadsGrant McAlister · PostgresOpen Silicon Valley 2017
• 32mLoading at Hyperscale: Advanced Data Import Techniques for CitusColton Shepard · Citus Con: An Event for Postgres 2022
• 31mDemanding the impossible rigorous database benchmDmitry Dolgov · DevConf.CZ 2023
• 18mBest Practices for Connection Management and Memory AllocationRon Xing, Harry Huang · Distributed SQL Summit Asia 2024
• 36mAutomated Performance Testing of PostgresGreg Burek · PostgresOpen 2014