Data warehouse and Kubernetes: Lessons from ClickHouse Operator
summary
An operator that deploys ClickHouse on Kubernetes actually provisions StatefulSets, persistent volume claims, and a Zookeeper ensemble from a single custom resource and templates, e.g. a 2x2 replicated cluster. Upgrades run as sequential rolling pod replacements by switching container version (19.6 → 19.11). Monitoring uses Prometheus + Grafana. Measured results show faster deploy times and virtually no query overhead versus comparable setups, CloudFlare-scale ingestion (≈10M events into a 100‑node cluster) is supported. Known failure modes: DNS caching and storage misconfiguration can cause data loss. ClickHouse Operator is available for use.
AI-written summary · about
chapters (10)
- 0:00Introduction and welcome
- 1:44Why run data warehouse on Kubernetes
- 3:09ClickHouse data warehouse overview
- 7:22Kubernetes basics and container management
- 12:14Introducing ClickHouse Kubernetes Operator
- 14:43Configuring ClickHouse clusters in Kubernetes
- 21:29Live demo of ClickHouse Operator
- 25:42Key lessons and challenges learned
- 29:36Future of data warehouses and Kubernetes
- 31:45Q&A session
related talks
27mStateful Workloads at Scale – Extending Kubernetes for Running OLAP Databases as a Service.
55mBuilding Fast ClickHouse Analytics for MySQL Transaction Data Using Kubernetes
33mIs It Safe? Security Hardening for Databases Using Kubernetes Operators
17mBuilding Real-Time Data Lake with ClickHouse
51mWhat You Need to Know About ClickHouse Architecture
49mSolving real business cases with ClickHouse
59mEffective Use of ClickHouse
35mOrchestrating Resilient Data: Harnessing the Strength of Kubernetes with Operators