Event-Driven Change Data Capture using Apache Pulsar

on YouTube ↗50m

" summary

Describes implementing log‑based CDC by wiring Cassandra (Astra DB) to Apache Pulsar: enabling CDC creates two topics per table (data + lock), Pulsar persists via uses Apache BookKeeper and separates compute and storage, and offers lightweight transformation with Pulsar Functions. Demonstration shows creating an Astra streaming tenant, topic and enabling CDC; connectors exist to push changes to sinks like Elasticsearch but the live sink test is not completed. Pulsar claims guaranteed delivery and a managed Astra plan with $25 free monthly credit.

AI-written summary · about

" chapters (8)

1. 0:00Introduction
2. 1:09Pulsar and DataStax intro
3. 3:15Change data capture importance
4. 8:49Modern CDC requirements
5. 15:51Pulsar features
6. 26:26CDC with Astra Streaming and Astra DB
7. 31:40Live CDC setup demo
8. 46:47Resources and closing

" related talks

• 36mEvent-Driven Change Data Capture patternMary Grygleski · Conf42 Cloud Native 2023
• 48mApache Pulsar: Event Streaming in The Cloud Native WorldMary Grygleski · GOTO Chicago 2023
• 53mEvent Messaging and Streaming with Apache PulsarMary Grygleski · Build Stuff 2022
• 32mStreaming Analytics with Apache PulsarAddison Higham · Aerospike Digital Summit 2021
• 47mEvent Streaming for the Best of All WorldsMary Grygleski · Spring I/O 2022
• 51mEvent Streaming and Processing with Apache PulsarMary Grygleski · Conf42 Cloud Native 2022
• 48mBest of Both Worlds: Apache Pulsar and Apache KafkaMary Grygleski, Ko Turk · Devnexus 2023
• 31mThe Power of Apache Pulsar. Harnessing Dapr to Build High Scale MessagingHugo Smitter · AppDeveloperCon NA 2024