Examining the Principles of Observability and Its Relevance in LLM...
summary
Describes how an OpenTelemetry collector plus Traceloop instrumentations capture per-query fields like vendor prompt temperature accuracy model_version and metrics for latency, tokens and cost, exportable to DataDog New Relic Instana or Zipkin. Shows a Docker Compose demo wiring a sample app to Jager/Zipkin and Prometheus, enabling aggregation of traces to spot hallucinations, measure cost per prompt and drive a fine‑tuning loop. Emphasizes trace/metric/log pipelines and supported LM providers and vector DBs.
AI-written summary · about
chapters (13)
- 0:00Observability and LLMs
- 1:47Current landscape of LLMs and applications
- 3:12Why AI observability matters
- 4:18LLM observability tools
- 5:47OpenTelemetry AI standards
- 6:51Trace Loop open source
- 8:19OpenTelemetry collector explained
- 10:05Full stack AI observability
- 12:15Fine-tuning with observability
- 12:37Watson X with Jaeger demo
- 14:24Analyzing LLM data
- 15:47Prompt and cost aggregation
- 16:29Q&A
related talks
32mFrom Traces To Action: Auto-Instrumenting LLMs for Observability...
02h 02mHow To Troubleshoot and Monitor Applications using OpenTelemetry
23mModern Application Debugging: An Intro to OpenTelemetry
22mOpenTelemetry – An Observability Framework for Cloud-Native Software
33mPractical introduction to OpenTelemetry tracing
55mCloud-Native Observability with OpenTelemetry
20mObservability and Open Telemetry
36mObservability For You and Me with OpenTelemetry