CDC Streaming Patterns in Distributed Systems
summary
Overview of change data capture with Debezium 2.0 and concrete patterns: shows connectors for MySQL, PostgreSQL, Oracle log miner and a new JDBC sync connector shipping this month, plus a Debezium Server with ~dozen adapters to Kinesis Pulsar Pub/Sub Event Hubs. Demonstrates the outbox table implementation that Debezium reads from transaction logs and writes to Kafka, the Strangler fig migration approach with bi‑directional CDC pipelines and Kafka Connect transformations for schema changes, and combining outbox plus CDC to implement compensating steps in a Saga orchestration used in production at Reddit Shopify and Ubisoft.
AI-written summary · about
chapters (5)
- 0:00Introduction to CDC and Debezium
- 13:17Outbox pattern with CDC
- 17:34Strangler fig pattern explained
- 26:40Saga pattern in distributed systems
- 31:57Q&A and closing remarks
related talks
20mChange Data Capture in Distributed Systems
51mGeo-Distributed Postgres Replication Using Debezium
53mBuilding Event Driven Services with Apache Kafka and Kafka Streams
50mEvent-Driven Change Data Capture using Apache Pulsar
02h 18mDistributed Systems: In the Daily Life of the Data Engineer
34mReal-Time Supply Chain Analytics Pipeline at X5 Based on Debezium CDC and Flink
22mJoining Dozens of Data Streams in Distributed Stream Processing Systems
45mSEDA + Kafka in large-scale streams: Via Varejo's Black Friday case