Enhancing Asynchronous Communication Observability with OpenTelemetry
Talkat Observability Day North America 2024 · on Tuesday, 2024-11-12· at 14:05· Salt Palace | Level 2 | 255 B·
summary
Demo shows OpenTelemetry plus the Messaging SemConv and an OpenTelemetry Collector JMX receiver or Prometheus JMX agent pulling Kafka broker metrics, then correlating those with client spans in a ClickHouse-backed store using SQL to join on client_id, service.instance.id, service.name. A query surfaces 2.4k of 43k spans exceeding 10ms for a producer–consumer pair. Recommendations include keeping metrics low cardinality, using span links by default with optional parent–child to control trace size.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 1:24Messaging challenges
- 4:14Kafka telemetry limits
- 6:24OpenTelemetry for messaging
- 8:53Messaging metrics conventions
- 13:30Demo: spans and Kafka metrics
- 17:27Future of messaging observability
- 18:29Q&A
related talks
26mThe Art of Event Driven Observability with OpenTelemetry
41mTracing event-based systems with OpenTelemetry
38mPractical introduction to OpenTelemetry tracing
30mBuilding Observable Streaming Systems with OpenTelemetry
35mOpenTelemetry for Busy People
33mObservability-First Kafka: Engineering Visibility at Scale
57mYour Distributed System is Talking to You: OpenTelemetry
27mAchieving End-to-end Visibility into Complex Event-sourcing Transactions using Distributed Tracing