Change Data Streaming Patterns in Distributed Systems
summary
Shows how Debezium plus Kafka Connect or the embedded engine capture DB transaction logs to implement three practical patterns: reliable event exchange via an outbox table written in the same DB transaction and routed with Debezium’s outbox router, gradual migration using CDC + Kafka + Debezium + Apache Flink to build aggregates (owners+pets) in-flight, and CDC-based saga orchestration with a saga execution log table and compensating transactions. Includes connector configs, Single Message Transforms, a RocksDB state store example to enrich Postgres “toast” partial events, and a working GitHub repo demonstrating the flows.
AI-written summary · about
chapters (6)
- 0:00Introduction
- 1:45Debezium — Log-based Change Data Capture
- 8:45Outbox Pattern
- 14:44Strangler Fig Pattern
- 26:47Saga Pattern
- 38:40Takeaways
related talks
30mChange Data Streaming Patterns in Distributed Systems
36mChange Data Streaming Patterns for Distributed Systems
42mData Streaming for Microservices using Debezium
37mData Contracts In Practice With Debezium and Apache Flink
49mChange Data Capture Pipelines with Debezium and Kafka Streams
01h 09mOpen-source Change Data Capture With Debezium
31mChange Data Capture for non relational databases
23mChange Data Capture: A RabbitMQ and Debezium Approach