Deep Dive on Using YugabyteDB's Change Data Capture Connector

on YouTube ↗39m

" summary

YugabyteDB exposes a built-in stateless CDC service that reads the WAL and decodes changes via the Postgres catalog, emits insert/update/delete flags, and maps tablets to Kafka partitions; a customized Debezium connector to Kafka runs as Docker or JAR and supports MSK/Redpanda/EventHub. Demos show an IoT stream streamed IoT -> Pinot demo with real-time aggregations and a JDBC sink where JDBC sink upsert/delete worked to keep Postgres in sync; connector and connector metrics are available.

AI-written summary · about

" chapters (8)

1. 0:00YugabyteDB CDC intro
2. 7:31CDC architecture
3. 13:35CDC integration strategies
4. 17:47CDC connector setup
5. 22:08IoT data to Kafka demo
6. 25:48Data to Pinot demo
7. 30:02PostgreSQL sync demo
8. 38:23Metrics and conclusion

" related talks

• 22mChange Data Capture: Exploring YugabyteDB’s Modern DesignPrasad Radhakrishnan, Suranjan Kumar · Distributed SQL Summit 2022
• 01h 09mOpen-source Change Data Capture With DebeziumGunnar Morling · Vaccination Database Tech Talks (Booster) - Spring 2022
• 18mData Modernization Using Confluent and YugabyteAnanda Bose · Distributed SQL Summit Asia 2022
• 30mAccelerate Modernization with Confluent and YugabyteDBAnanda Bose, Amey Banarse · Distributed SQL Summit 2022
• 35mChange Data Capture: The Magic Wand We Forgot #bbuzzMartin Kleppmann · Berlin Buzzwords 2015
• 19mChange data capture with Debezium…and withoutPetros Angelatos · Berlin Buzzwords 2022
• 48mStreaming Database Changes with DebeziumGunnar Morling · Devoxx Belgium 2017
• 49mChange Data Capture Pipelines with Debezium and Kafka StreamsGunnar Morling · Developer Summit 2020