Build your next-generation stream platform based on Apache Pulsar
summary
Apache Pulsar provides a multi-tenant pub/sub with separated compute and storage, BookKeeper-backed layered storage, and geo-replication; claimed single-partition throughput of 1.8 million messages/sec and durable, replayable logs. Integration with Apache Flink adds a schema registry, exactly-once source and sink, table API support, and a key-share subscription for key-based ordering. The stack lets clients read recent segments from brokers or older segments from deep storage (S3/HDFS) and outlines using Pulsar transactions and layered storage as a state backend.
AI-written summary · about
chapters (15)
- 0:00Introduction and speaker background
- 1:04What is Apache Pulsar
- 2:31Evolution of Apache Pulsar
- 5:03Unified data processing with Pulsar
- 7:33Cloud-native architecture of Pulsar
- 10:29Storage tiers and data retention
- 11:54Reading APIs and subscription models
- 13:16Pulsar and Flink integration
- 14:44Pulsar schema registry
- 17:57Flink Pulsar source API
- 18:40Schema-specific data consumption
- 19:21Exactly-once processing in Flink
- 20:28Future directions and improvements
- 21:59Conclusion and community resources
- 23:04Q&A session
related talks
45mScalable distributed messaging&streaming with Apache Pulsar
53mEvent Messaging and Streaming with Apache Pulsar
48mApache Pulsar: Event Streaming in The Cloud Native World
47mEvent Streaming for the Best of All Worlds
45mApache Pulsar: Finally an Alternative to Kafka?
46mScalable messaging and streaming with Apache Pulsar
50mApache Pulsar 101: architecture and features
51mApache Pulsar 101: architecture, concepts & benchmarks