Primer (...)
summary
Primer is a distributed, persistent message queue implemented as an Erlang OTP application that is peer‑to‑peer (full mesh), supports PUSH and PULL, optional disk-backed retention with ranged logs, and consumer groups with checkpoints. Design specifics include gossip membership (Plumtree/SWIM), best‑effort ordering, configurable sync replication and chain replication, and local queues with acknowledgements. It is already used in production and claims support for millions of messages and hundreds of thousands of clients; source planned to be released April 2015.
AI-written summary · about
chapters (14)
- 0:00Primer introduction
- 1:12Creating a new message queue
- 3:19CouchDB inspiration and use cases
- 6:12Master-master architecture
- 9:31Primer as distributed system
- 13:53Message distribution and replication
- 21:33Replicated logs and chain replication
- 24:05Dynamic queue ranges and sharding
- 31:13Consumer groups and consumption
- 32:15Persistence and memory management
- 34:20Message mapping and routing
- 36:09Monitoring metrics and queues
- 36:52Future plans and open source
- 37:15Production usage and performance
related talks
40mPerformance Improvements
41mConsistent, Distributed Elixir
24mBarrel, keep data in sync in your Erlang application
19mBuilding a Distributed Data Ingestion System with RabbitMQ
03mBuilding a Distributed Data Ingestion System with RabbitMQ
43mBuilding a P2P Multi-Purpose Database
01h 01mBeyond Pub/Sub - Advanced Messaging Patterns
17mMigrating Primer’s Live Payments Platform to CockroachDB with Zero Downtime