FlinkCDC: Streamlining your data analytics pipelines
summary
FlinkCDC is an Apache Flink component that reads DB binlogs (MySQL/Postgres via an embedded Debezium client or native MongoDB readers), applies an incremental snapshot reading algorithm to avoid table locks, and provides exactly‑once, fault‑tolerant capture plus schema evolution. Version 3.0 adds a YAML end-to-end pipeline for source→transform→sink jobs; demoed joins across MySQL and Postgres with real‑time updates and a live sync into Elasticsearch, showing the pipeline runs and changes propagate correctly.
AI-written summary · about
chapters (9)
- 0:00Introduction to Flink CDC
- 1:25What is Change Data Capture
- 6:24Common CDC use cases
- 7:29Introducing Flink CDC
- 9:35Flink CDC internals
- 13:34Flink CDC 3.0 updates
- 15:02Flink CDC APIs and architecture
- 20:52Demo streaming ETL
- 35:01Conclusion and Q&A
related talks
42mChange Data Capture and Processing with Flink SQL
27mUnderstanding stream table duality using Kafka, Flink SQL and Debezium format
39mChangelog Stream Processing with Apache Flink
30mChange Data Capture With Flink SQL And Debezium
43mContinuous SQL with Kafka and Flink
46mFlink SQL in 2020: Time to show off!
41mBeyond the Hype: When Apache Flink Solves Real Problems
42mApache Flink deep-dive #bbuzz