Streams: past, present, and future
summary
A dedicated replicated log library called Osiris replicated log replaces the Raft prototype and uses leader/quorum writes, a dedicated TCP sendfile path and relaxed fsync semantics to hit 10–20× throughput versus the original implementation; streams are stored as segment files with dynamic chunks that carry index and trailer metadata and support chunk-level Bloom filters so the broker can skip chunks and implement stream filtering in 3.13 (examples show ~29% and ~80% bandwidth savings depending on message size and ingress rate).
AI-written summary · about
chapters (7)
- 0:00RabbitMQ streams intro
- 2:07Designing RabbitMQ streams
- 10:08Streams internals and performance
- 16:31Scaling with super streams
- 22:27Filtering in RabbitMQ 3.13
- 33:29Future plans and community
- 36:20Q&A
related talks
42mIt's like 10,000 streams when what you need is a queue
38mStream-style messaging development
36mStream All The Things!
27m"Write Once Stream Everywhere: Transforming Your Data into Events"
48mSpring for Rabbit Super Stream and SQL filters
37mRaft - The Understandable Distributed Protocol
55mThe data dichotomy: Rethinking data and services with streams
30mEvent Streaming and the Future of Applications