PostgreSQL Distributed & Secure Database Ecosystem Building

on YouTube ↗42m

" summary

Claim: transform existing PostgreSQL into a sharded, compute/storage–split database using Apache ShardingSphere, deploying stateless distributed SQL compute nodes on Kubernetes while leaving storage on existing Postgres or RDS. Tools: JDBC driver and proxy, Helm charts, demo with three compute instances. Features: elastic compute scale‑out, transparent sharding table creation, automatic data migration, column encryption and SQL audit. Project is mature (open source ~5 years, >14 releases) and the demo shows sharding and encryption actually working end to end.

AI-written summary · about

" chapters (16)

1. 0:00Introduction
2. 1:20Motivation for distributed clusters
3. 3:10Challenges in DB infrastructure
4. 6:32Monolith to distributed evolution
5. 8:00Compute-storage separation
6. 9:26PostgreSQL on Kubernetes
7. 12:00Benefits of ShardingSphere
8. 17:21ShardingSphere overview
9. 19:10ShardingSphere clients
10. 24:18Architecture before/after ShardingSphere
11. 26:06Distributed SQL for shards
12. 29:19Demo setup on Kubernetes
13. 33:15Sharded tables and encryption
14. 36:56Security and encryption
15. 37:59Summary
16. 39:01Q&A

" related talks

• 31mPostgreSQL Distributed & Secure Database Ecosystem BuildingTrista Pan · Conf42 Cloud Native 2022
• 27mAutomatically shard & scale-out traditional databases on K8sTrista Pan · Conf42 Cloud Native 2023
• 53mBreakout SessionJason Petersen · PGConf Silicon Valley 2015
• 45mThe Future of Postgres ShardingBruce Momjian · HighLoad++ 2015
• 45mThe Future of Postgres ShardingBruce Momjian · HighLoad++ 2015
• 38mPostgreSQL as a distributed computing platformMarco Slot · PostgresOpen Silicon Valley 2017
• 42mPostgreSQL Distributed: Architectures & Best practicesMarco Slot · PostgreSQL Conference Europe 2023
• 33mFive Sharding Data Models and Which is RightCraig Kerstiens · RailsConf 2018