Learnings, patterns and Uber’s metrics platform M3...
summary
Uber runs high‑cardinality monitoring across 4,000 microservices and puts 6 billion+ time series to work for auto‑discovery, automatic alert creation/teardown and rollups that collapse related failures into contextual alerts. The stack uses Prometheus plus M3DB as Prometheus long‑term storage to keep petabytes of metrics while supporting massive ingest — about 30M metrics/sec and ~50 Gbps pre‑replication — an index of ~9 billion metric IDs, and ~150,000 real‑time alerts with nodes handling up to ~500k writes/sec.
AI-written summary · about
chapters (13)
- 0:00Introduction and context
- 1:09Reusable monitoring software
- 2:35Metrics vs Nagios checks
- 5:50High dimensionality in metrics
- 9:28Uber’s metrics platform M3
- 12:45Centralized metrics at Uber
- 17:02M3’s scale and performance
- 19:52Cassandra to M3DB transition
- 24:08M3DB architecture
- 27:02M3DB design principles
- 30:12M3 as Prometheus storage
- 31:38Future of M3 and community
- 37:24Q&A
related talks
37mM3 and Prometheus, Monitoring at Planet Scale for Everyone
11mAggregating, Alerting and Graphing on Millions of Prometheus T...
33mHow Uber Uses Cassandra to Power Core Applications at Scale?
35mPlatformizing OpenSearch at Uber
35mDoorDash’s Journey From StatsD To Prometheus With 10 Million Metrics/Second
40mMeasuring and Optimizing Kubeflow Clusters at Lyft
31mHandling Billions of Metrics with Prometheus and Thanos
44mScaling OpenSearch at Uber: Operating Resilient, Fault-Tolerant, High-Performance Clus...