Streaming Database Changes with Debezium
summary
Debezium captures every row insert/update/delete by reading the database binlog / write‑ahead log and publishing change events to Kafka, with an initial snapshot then log‑tailing to ensure completeness. It runs as Kafka Connect source connectors for MySQL, PostgreSQL and MongoDB, serializes with Avro + Schema Registry, supports SMTs to reshape or route messages, and demonstrates a CRUD app -> Kafka -> Elasticsearch/Postgres pipeline that tolerates connector downtime and reliably catches up. Current release series is 0.6.x with 0.7 planned.
AI-written summary · about
chapters (15)
- 0:00Intro
- 0:53Gunnar Morling
- 1:46Change Data Capture what is it about?
- 2:36CDC Use Cases
- 8:41How to Capture Changes?
- 12:34Database Logs A closer loos
- 21:08CDC Topology with Kafka Connect
- 22:36CDC Message Structure
- 25:51Initial Snapshots
- 27:38Debezium Connectors
- 30:35Operating a Connector
- 37:25Message Transformations Altering individual messages via SMTS
- 43:23Trying It Out Yourself
- 46:18Summary
- 47:28Resources
related talks
42mData Streaming for Microservices using Debezium
01h 09mOpen-source Change Data Capture With Debezium
41mChange Data Streaming Patterns in Distributed Systems
31mChange Data Capture for non relational databases
49mChange Data Capture Pipelines with Debezium and Kafka Streams
52mChange Data Streaming For Spring Boot With Apache Kafka and Debezium
26mDebezium Server - New CDC runtime on the horizon!
30mChange Data Capture With Flink SQL And Debezium