Pulsar Beam, HTTP streaming over Apache Pulsar
summary
A Golang service exposes an HTTP POST for producing to Pulsar topics, an HTTP SSE interface for streaming consumption and a webhook-based push for deliveries to Lambda/Azure/GCP functions; components are stateless and independently deployable with a Helm chart and ~40MB Go images. It preserves Pulsar semantics (exclusive/shared/failover/key_shared subscriptions, batching, sync/async send, dead‑letter retries), maps message IDs through SSE, and demos show round‑robin delivery, replay from retention and webhook retries working end to end.
AI-written summary · about
chapters (16)
- 0:00Intro
- 2:20Message exchange patterns
- 6:36What is Apache Pulsar
- 7:47Apache Pulsar Components
- 11:00Message Pub/Sub
- 14:35Subscriptions and consuming messages
- 15:33Subscription types
- 24:10Pulsar Protocol
- 28:48Pulsar Clients
- 30:53Websocket
- 33:25HTTP Long Polling to consume messages
- 36:20SSE Consideration
- 36:38Produce messages
- 37:53Webhook notification
- 40:04Webhook components
- 41:25Pulsar Beam components
related talks
34mEvent Streaming in the Cloud Native World with Apache Pulsar
40mKeynote: Apache Pulsar
20mStreaming Processing Options in Google Cloud
36mHow to Build a Scalable Serverless IoT Architecture on GCP
51mJuggling 300 integrations while scaling to 10mil. Signals per day
05mIgnite: Enabling multi-cloud and breaking vendor lock-in with Cloud Sidecar
55mServerless Design Patterns for the Enterprise with AWS Lambda
03h 02mAPAC Live Stream