It's like 10,000 streams when what you need is a queue
summary
Compares concrete tradeoffs between Kafka (distributed log) and queue-based brokers like RabbitMQ/AWS SNS+SQS and Azure Service Bus, showing who tracks offsets, how partitioning and consumer groups limit autoscaling, and how dead messages cause head-of-line blocking unless routed to a dead letter queue. Recommends event stores for private event sourcing, queues for guaranteed workflow delivery, and streams plus Kafka Connect/Flink for analytics or forward caches, with examples and code on GitHub.
AI-written summary · about
chapters (14)
- 0:00Introduction to irony
- 1:50Event-driven basics
- 4:12Publisher-subscriber models
- 4:57Queues versus streams
- 8:55Message distribution state
- 12:15Consumer scale-out
- 17:02Message processing failures
- 21:08Message routing topologies
- 23:47Semantic routing
- 25:24Broker feature comparison
- 27:08Event sourcing vs streaming
- 30:40Domain events categories
- 32:44When to use which technology
- 39:53Summary and conclusion
related talks
36mStream All The Things!
50mMiddleware Showdown: Exploring Diverse Messaging Solutions
55mImplementing scalable messaging patterns in Azure
47mAdopting Polyglot Flow
46mReal-world Examples of Message Queue Usage: Insights from RabbitMQ
01h 03mChoosing the Right Messaging Solution for Your Architecture
30mEvent Streaming and the Future of Applications
47mStream All the Things!