Managing Quorum Queues at Scale
summary
Amazon MQ reports that RabbitMQ 3.13 adds continuous membership reconciliation to keep a queue’s replication group at a configured target size automatically, and a permanent voter status so newly-joined nodes don’t count for majority until they catch up. Operators use the quorum-critical check plus management API fields (state: running/minority/down) and Prometheus metrics (apply_index vs commit_index) to detect stuck followers. Recovery tools include maintenance/drain, add/remove member commands, an internal force-shrink that bypasses majority, and regular backups.
AI-written summary · about
chapters (17)
- 0:00Managing quorum queues
- 1:07Operating at Amazon’s scale
- 2:33Quorum queues and Raft
- 4:42Node failures in quorum queues
- 5:45Automating membership
- 7:10Health checks for queues
- 9:15Critical check for shutdowns
- 12:05Monitoring availability
- 14:11Stuck members detection
- 16:37Summary of health checks
- 17:18Recovery strategies
- 20:55Backup and disaster recovery
- 21:18Recap of managing queues
- 22:02Q&A testing verification
- 24:33No leader handling
- 26:40Q&A voter status checks
- 28:47Q&A maintenance mode
related talks
18mHow to Break RabbitMQ
35mHow to Safely and Reliably (...) While Maintaining 100% Uptime
21mWhy does RabbitMQ stop and what can be done about it
39m101 ways to break your RabbitMQ
22mLessons Learned from Scaling RabbitMQ in a High-Growth (...)
34mUpdate RabbitMQ now!
40mScaling RabbitMQ
48mA Beginner's Guide to Inter-Service Messaging