Event-Driven Change Data Capture Pattern Using Apache Pulsar

on YouTube ↗46m

" summary

Shows a working CDC pipeline that reads Cassandra change logs, pushes events into Apache Pulsar via DataStax Astra Streaming, applies lightweight transformations with Pulsar Functions, and sinks records to targets like Elasticsearch using Pulsar connectors. Demo uses the Astra UI to create tenants, topics and producers so CDC topics appear automatically; storage is handled by BookKeeper and cluster metadata by ZooKeeper. Project growth even outpaces Kafka contributors as of 2021, and the demo components run end-to-end.

AI-written summary · about

" chapters (11)

1. 0:00Start
2. 2:05Agenda
3. 11:28Log Based Cdc
4. 12:56Requirements for a Modern Uh Cdc System
5. 14:0412 Factor Guidelines
6. 15:28Scalability
7. 21:31What Is Pulsar
8. 28:54Components of Pulsar 2
9. 29:16Components of Pulsar
10. 30:28Zookeeper
11. 35:15Enabling Cdc

" related talks

• 36mEvent-Driven Change Data Capture patternMary Grygleski · Conf42 Cloud Native 2023
• 48mApache Pulsar: Event Streaming in The Cloud Native WorldMary Grygleski · GOTO Chicago 2023
• 34mEvent Streaming in the Cloud Native World with Apache PulsarMary Grygleski · Cloud Builders: Java Conf 2023
• 55mEvent Messaging and Streaming with Apache PulsarMary Grygleski · Great International Developer Summit 2023
• 51mEvent Streaming and Processing with Apache PulsarMary Grygleski · Conf42 Cloud Native 2022
• 59mEvent Streaming and Processing Using Apache PulsarMary Grygleski · IntelliJ IDEA Conf 2022
• 30mChange Data Streaming Patterns in Distributed SystemsGunnar Morling, Hans-Peter Grahsl · Berlin Buzzwords 2021
• 35mStreaming Data Made EasyTimothy J Spann, David Kjerrumgaard · OSA Con 2022