How to Efficiently Manage Logs in Large-Scale Kubernetes Clusters
summary
Compares ElasticSearch, Grafana Loki and VictoriaLogs with concrete benchmarks: claims 10x+ infrastructure savings, shows a real case where VictoriaLogs replaces a 27-node ElasticSearch cluster (≈600 CPU cores, 4–6 TB RAM) with a single node on 8 cores/64 GB that serves production. Staging instance stores 2.54 billion logs (3.4 TB uncompressed) compressed to 46 GB (~70x), uses ~650 MB RAM, answers “error” queries for a day or 100 days in <200 ms, a top-5 analytics query in ~400 ms, and finds rare tokens across 3.4 TB in ~3 s using ~3 GB of bloom filters; object-storage and deletion/masking features are planned.
AI-written summary · about
chapters (14)
- 0:00Kubernetes log management intro
- 1:10Cluster architecture & log sources
- 4:26kubectl logs limitations
- 5:53Why large-scale logging
- 7:20Centralized log DB requirements
- 9:53Traditional vs analytical DBs
- 12:18Document indexing databases like Elasticsearch
- 13:21Specialized DB drawbacks
- 15:28VictoriaLogs introduction
- 18:01VictoriaLogs performance metrics
- 23:26High query performance
- 29:35VictoriaLogs for Kubernetes
- 30:40Next steps with VictoriaLogs
- 31:01Q&A on VictoriaLogs
related talks
40mLarge-Scale Logging Made Easy
40mScale Your Monitoring Solution With the VictoriaMetrics Ecosystem
36mVictoriaMetrics: scaling to 100 million metrics per second (EN)
28mFrom Fragmented Logs to Unified Insights: Scaling O11y
28mHow to reduce expenses on monitoring with VictoriaMetrics
40mTaming Spiky Log Volumes: Maintaining Real-Time Log Accessibility with Kaldb
46mConquering Distributed Cloud Logging Without Breaking the Bank
30mCentralized Log Management