RabbitMQ-less OpenStack Cluster Supporting 14,000 HVs in One...
summary
Claim: an RabbitMQ-less OpenStack RPC backend replaces RabbitMQ with a custom HTTP driver plus Consul service discovery and a broadcaster, removes the messaging broker SPOF and scales a Nova cluster to 14,000 hypervisors in one region. Implementation details: servers register endpoints in Consul, clients fetch targets and call HTTP, broadcaster fan‑out handles cast semantics. Deployed two years in production with far fewer incidents, though operators warn about lost rehearsal experience and recovery practices.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 1:03Belud private cloud overview
- 2:51Large-scale OpenStack bottlenecks
- 5:48RPC call and cast in messaging
- 7:40RabbitMQ problems at scale
- 9:51Stabilizing RabbitMQ attempts
- 12:19HTTP driver project goals
- 13:23OpenStack messaging drivers
- 17:02HTTP driver architecture
- 20:12RPC call and cast in HTTP driver
- 24:36Broadcasting in HTTP driver
- 26:43HTTP driver in production 2 years
- 27:49RabbitMQ message loss
- 30:19Stable RPC backend lessons
- 32:04HTTP driver FAQ
- 34:18HTTP driver upstream progress
- 35:11Q&A
related talks
19mBuilding a Distributed Data Ingestion System with RabbitMQ
35mMigrating from Kafka to RabbitMQ at SimpleBet: Why and How
31mRabbitMQ on Kubernetes Deep Dive
40mPerformance Improvements
27mHow to get rich with cloud!
22mEffective migration from Newton to Queens with minimal downtime
48mBuilding a Distributed Data Ingestion System with RabbitMQ
32mRabbitMQ in Real Life - Tips and Tricks