Cloud Native Distributed Event Streaming from TiKV
summary
Produces a deduplicated, commit-timestamp-ordered change stream from TiKV by capturing events at the per-region Raft/replication layer and emitting watermarks once PD’s logical commit ts and region logs show no earlier commits, enabling consistent transactional ordering across thousands of moving regions (nodes can host >10,000 regions). Implements the streamer as a separate service TiCDC that writes to Kafka/S3/MySQL, uses etcd-backed HA and persistent state for stateless workers and leader election, and reports low latency and no single point of failure.
AI-written summary · about
chapters (4)
related talks
11mAdapting TiKV for Cloud Storage
49mThe Paradigm Shift - Understanding the Move To Distributed Databases
29mDistributed SQL: How TiDB Modernized Data Management
28mHow Does Geo-Replication Work in TiDB?
29mHow We Make TiKV- a Distributed Storage, More Cost-Effective On the Cloud
29mDistributed SQL How TiDB Modernized Data Management
40mTiKV Best Practices
27m"Write Once Stream Everywhere: Transforming Your Data into Events"