Replicated Subscriptions: taking Apache Pulsar Geo-Replication to next level
summary
Replicated subscriptions in Apache Pulsar (introduced around 2.6 and matured by 2.8) keeps subscription offsets in sync across clusters by injecting marker messages over the replication data path and building a cross-over snapshot that maps local message IDs to peer positions. Snapshots run by default every 1 second (tunable) so failover resumes within subsecond to one-second drift, costs tiny extra control traffic, works for 2–100 clusters, uses a two‑round protocol for 3+ clusters, and requires all clusters reachable to create a snapshot.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 1:56Geo-replication fundamentals
- 7:35Common replication patterns
- 11:28Why replicated subscriptions
- 13:34How replicated subscriptions work
- 27:04Tuning and trade-offs
- 27:45Benefits and use cases
- 36:21Q&A
related talks
46mApache Pulsar: Finally an Alternative to Kafka?
10mA tour of Apache Pulsar
51mEvent Streaming and Processing with Apache Pulsar
50mApache Pulsar 101: architecture and features
45mApache Pulsar: Finally an Alternative to Kafka?
22mBuilding FinTech Streaming Applications with Pulsar
31mThe Power of Apache Pulsar. Harnessing Dapr to Build High Scale Messaging
27mLeveraging Event Streaming to Super-Charge your Business Solutions.