Warp-Speed Debugging with Prometheus Exemplars
Talkat PrometheusDay Europe 2022 · on Tuesday, 2022-05-17· at 14:40· Pavilion 3, Room F | Level 2 | Central Forum + Live Stream, Feria Valencia, Spain + Live Stream·
summary
Prometheus exemplars attach user-supplied labels to metric samples so Grafana can offer one‑click jumps from a metric point to traces (Tempo) or profiles (Parca), shown as green triangles on a percentile graph; exemplars use the OpenMetrics format, Prometheus stores them in an in-memory circular buffer controlled by max_exemplars (only the most recent exemplar per label set survives between scrapes), they’re experimental and must be enabled, and Grafana maps them via internal/external links to provide the claimed warp-speed hops between data streams.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 0:31About Ian Billett
- 1:32Agenda
- 4:13Time to Detection Time to Recovery
- 6:49Improving Debugging Efficiency
- 10:43Creating Exemplars
- 11:47Exemplars in Prometheus
- 14:43Prometheus Exemplars
- 16:46Prometheus Exemplars Query
- 17:46Prometheus Exemplars Data Sources
- 19:53Demo
- 24:02Questions
related talks
15mThe "Zen" of Python Exemplars
27mObservability with Prometheus and beyond
35mAn Exemplary Path: Leveraging EBPFs and OpenTelemetry To Auto-instrument for Exemplars
36mObservability with Prometheus and Beyond: Leveraging Cloud-native Technologies Outside of Cloud
31mEvolving Prometheus for More Use Cases
36mHow to Use Prometheus's Native Histograms
32mDemocratizing Application Monitoring with Prometheus
24mExemplars in Practice: Finding the Needle in Your Observability Haystack