A Change-Data-Capture use-case: designing an evergreen cache
summary
Claims that Change-Data-Capture keeps a cache evergreen by wiring database binlog events directly into the cache rather than using TTLs or fragile triggers. Implementation uses Debezium connectors to read MySQL/Postgres binlogs and a Hazelcast Jet pipeline that maps insert/update/delete events to cache operations and writes updates into Hazelcast cache in memory. Demo shows the pipeline catching changes that occurred before it started, avoids DB-side C++ hooks, and preserves existing applications and schemas.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:31Why cache data
- 10:20Cache invalidation challenges
- 16:21Change data capture basics
- 23:48Debezium for CDC
- 25:56Hazelcast Jet overview
- 29:45Evergreen cache design
- 33:13Demo setup
- 36:02Running the pipeline
- 39:18Summary and resources
related talks
32mA Change-Data-Capture use-case: designing an evergreen cache
32mA Change-Data-Capture use-case: designing an evergreen cache
35mChange Data Capture: The Magic Wand We Forgot #bbuzz
01h 09mOpen-source Change Data Capture With Debezium
20mChange Data Capture in Distributed Systems
23mChange Data Capture: A RabbitMQ and Debezium Approach
39mDeep Dive on Using YugabyteDB's Change Data Capture Connector
15mScaling PostgreSQL Change Data Capture Using Open Source Airbyte