Wargaming ElasticSearch Cloud Architecture
summary
Describes an Elasticsearch cloud design built on a HashiCorp stack (Consul, Nomad, Vault) using host networking and strict node pinning to keep data nodes colocated on physical hosts. Logs and metrics run outside Docker; multi‑tenant isolation is achieved by deploying separate ES clusters; monitoring is Prometheus + Alertmanager with Consul service discovery, and curator enforces index retention and disk quotas. The deployed system avoids containerizing core components, allocates Elasticsearch per node, and achieves measured peak ingestion up to 400k docs/s per cluster.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 3:12Existing solutions
- 5:42Nomad and Consul
- 10:00ElasticSearch Cloud
- 12:07Docker issues
- 16:32Traffic balancing
- 17:58Authorization
- 20:08Monitoring with Prometheus
- 24:02External monitoring
- 26:55Grafana and Kibana
- 27:39Cluster deployment
- 29:05Resilience testing
- 30:53Conclusions
- 31:49Q&A
related talks
32mArchitecting Geo-Distributed Mobile Edge Application With Consul
31mAn Adventure in Multi Cloud with Kubernetes and Distributed Data
28mAutomated Deployment of ElasticSearch on AWS
40mTo Build My Own Cloud with Blackjack…
40mBuilding a Modern Infrastructure Stack
57mWhat Is the Cloud?
01h 02mContainers in Production: It’s Like Orchestrating Cats
36mBuilding a Scalable Data Science & Machine Learning cloud using Kubernetes