Innovating on Top of Open Source Observability - Pushing the Boundaries o...
summary
eBay explains concrete engineering: a leaf/zone/root layered architecture that keeps Prometheus TSDB unchanged while adding an indexer, remote_write/read adapters and Kubernetes ring discovery to scale horizontally; a ClickHouse back-end for traces with many small clusters, trace-ID sharding and a custom OpenTelemetry receiver that pulls completed traces after a 5‑minute delay to generate service graphs and cut memory use; and exemplar-enriched alerts via a custom Alertmanager stage that adds trace IDs to annotations, speeding triage but exposing Alertmanager concurrency limits at 4,000 services.
AI-written summary · about
chapters (9)
- 0:00Open source observability challenges
- 1:01Open source customization at scale
- 2:49Scaling with public APIs and standards
- 4:36Prometheus scaling with Monarch
- 9:11ClickHouse for logs and traces at eBay
- 12:04Service graph with delayed span processing
- 14:14Trace ID routing for microservices
- 16:19Alerts enriched with trace IDs
- 19:50Community contributions
related talks
30mIs Building an Observability Platform with Open Source the Right Choice?
26mDistributed Tracing: All the Warning Signs Were Out There!!!
25mUnifying Observability: Weaving Prometheus, Jaeger, and Open Source Together to Win
37mBuilding Modern Observability Architecture With OpenSearch, OS Ingest...
41mNavigating the Open Source Observability Landscape
45mOpen Source for Better Observability
01h 02mDesigning an open-source observability stack for Kubernetes
13mArchitecting an open observability stack