Change data capture with Debezium…and without
summary
Materialize implements a direct Postgres CDC connector that skips Kafka+Debezium and reads Postgres logical replication, decoding the WAL into commit-ordered transactions. It snapshots with COPY inside a transaction that creates the replication slot created inside snapshot transaction so replay starts at the exact LSN with no gaps or duplicates. Every row event is timestamped with the commit LSN to preserve transaction atomicity. Implementation uses Rust postgres replication APIs, handles WAL retention feedback, and warns about slot-driven disk growth.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 0:27What is materialized
- 1:35Debezium architecture
- 3:59Why Debezium
- 5:40The rightahead log
- 11:52The replication slot
- 16:52Questions
related talks
01h 09mOpen-source Change Data Capture With Debezium
31mChange Data Capture for non relational databases
35mChange Data Capture: The Magic Wand We Forgot #bbuzz
23mChange Data Capture: A RabbitMQ and Debezium Approach
01h 11mChange data capture pipelines with Debezium and Kafka Streams
39mDeep Dive on Using YugabyteDB's Change Data Capture Connector
41mChange Data Streaming Patterns in Distributed Systems
20mChange Data Capture in Distributed Systems