How To Supercharge AI/ML Observability With OpenTelemetry and Fluent Bit
Talkat KubeCon + CloudNativeCon Europe 2025 · on Friday, 2025-04-04· at 15:15· Level 0 | ICC Capital Hall | Room J·
summary
Combines OpenTelemetry OTLP with Fluent Bit head/tail sampling to capture logs, traces and metrics from an EKS demo cluster with two GPU nodes running two LLMs (Llama 8B instruct and Llama 8B tulip). A plain Python app uses OpenTelemetry auto-instrumentation, Fluent Bit exposes OTLP input/output and forwards data to Chronosphere; p99 client latency and token usage (≈950 tokens) appear in the UI, and conditional filters drop noisy traces so the pipeline remains practical and effective.
AI-written summary · about
chapters (5)
- 0:00AI/ML observability challenges
- 3:53Observability dimensions in AI
- 7:58OpenTelemetry + Fluent Bit integration
- 12:55Demo: Monitoring LLM models
- 17:48Q&A
related talks
28mBuilding Observability Pipeline with Fluent-Bit
36mFluent Bit: Logs, OpenMetrics, and OpenTelemetry all-in-one
26mPlug-and-Play Observability: OTel Auto-Instrumentation for LLMs in OpenSearch
32mFrom Traces To Action: Auto-Instrumenting LLMs for Observability...
18mWhat did AI just do? Making LLMs Observable with OpenTelemetry
40mOptimizing AI Applications for Production with Observability
23mExporting your trace telemetry with OpenTelemetry and Fluent Bit
23mOpenTelemetry Auto Instrumentation for Effortless Observability