Logical Replication handling of large transactions
summary
Covers how PostgreSQL moves from PG12’s temp-file-per-large-transaction (publisher-side spill) to PG13’s memory cap logical_decoding_work_mem=64MB (default, min 64KB), PG14’s subscription streaming that sends chunked replication messages (publisher streams, subscriber may still spill), and PG16’s streaming=parallel which launches parallel apply workers on the subscriber to apply in-progress transactions immediately. Implemented change reduces apply latency and tests show about 40% faster elapsed time for large-batch inserts; deadlock avoidance, failure restart, and monitoring counters are also added.
AI-written summary · about
chapters (18)
- 0:00Intro
- 0:50Agenda
- 3:07Logical decoding | Workflow
- 5:12Logical replication | Usage
- 6:13Logical replication | Workflow
- 7:26Logical replication | Monitoring
- 9:27Decoding for large transaction (in PG12)
- 11:05and Improvements in PG13
- 13:04and Improvements in PG14
- 16:28Improvements in PG14 | Monitoring
- 19:14Overview and Advantage
- 21:46Architecture
- 24:15Basic workflow
- 25:20Commit protocol
- 27:22Failure Handling
- 30:35Consideration #1: Deadlock between the leader apply worker and the parallel apply worker
- 36:27Future development
- 37:35Summary
related talks
42mlesser known Superpower of Postgres Logical Replication
44mThe journey towards active-active replication in PostgreSQL
31mPast, Present, and Future: Logical Decoding and Replication in PostgreSQL
48mPostgreSQL Replication: 20 Pitfalls and Solutions
29mLogical replication theory and concepts
43mPostgreSQL 17 and beyond
01h 41mPostgreSQL Upgrades with Logical Replication & Minimal Downtime
54mUnderstanding Logical Decoding and Replication