Scalable distributed messaging&streaming with Apache Pulsar
summary
Apache Pulsar is a cloud‑native messaging and streaming platform that separates stateless brokers from durable storage (Apache BookKeeper), storing topics as segments replicated with a replication factor of three and allowing segments to offload to S3 for virtually unlimited history. It scales on three independent axes: add consumers via key_shared/shared subscriptions without broker changes, add broker nodes with no data rebalancing when scaling brokers, and add bookies for storage. Built‑in geo‑replication, multi‑tenancy, Pulsar Functions, connectors, and a Kafka protocol adapter enable migrations; one user reports cutting node count by 100.
AI-written summary · about
chapters (11)
- 0:00Introduction to Pulsar
- 2:19Data streaming fundamentals
- 5:16Message queuing vs. streaming
- 8:38Kafka for message queuing
- 11:44Pulsar unified platform
- 17:10Scalability and architecture
- 21:33Durability and storage
- 24:05Advanced Pulsar features
- 26:03Ecosystem and compatibility
- 28:54Community and summary
- 31:10Q&A session
related talks
46mScalable messaging and streaming with Apache Pulsar
53mEvent Messaging and Streaming with Apache Pulsar
28mBuild your next-generation stream platform based on Apache Pulsar
45mApache Pulsar: Finally an Alternative to Kafka?
51mApache Pulsar 101: architecture, concepts & benchmarks
50mApache Pulsar 101: architecture and features
47mChoosing the Right Messaging Platform for Your Event-Driven Application
48mApache Pulsar: Event Streaming in The Cloud Native World