Optimizing AI Applications for Production with Observability
summary
Shows how OpenTelemetry instruments AI apps end to end and proves it by wiring Python and .NET samples into a local Aspire dashboard (docker OTLP endpoint) and into Azure Monitor / Application Insights for production tracing. Demos emit traces, metrics and logs; example RAG pipeline records embeddings (921 ms), a chat completion (7 s of 11 s total), LangChain/LlamaIndex/Semantic Kernel integrations, and a Locust load test (10 users) that surfaces rate‑limit and token‑quota exhaustion tracked via token usage metrics and remaining‑requests counters.
AI-written summary · about
chapters (11)
- 0:00Introduction to session
- 1:26Concepts of observability
- 4:47Open telemetry architecture
- 8:00Traces, metrics, and logs
- 11:04Telemetry for AI ecosystems
- 14:15Aspire local dashboard demo
- 18:30Production with Azure monitor
- 22:15Observability in RAG apps
- 27:58Implementing AI observability
- 30:11Load testing and triaging
- 36:37Summary and resources
related talks
38mAI Monitoring Mastery: Optimize Performance, Ensure Compliance, and Cut Costs
31mDeep Dive To AI Agent Observability
01h 01mLogging, tracing and metrics: instrumentation in .NET and Azure
30mFrom Black Box To Insight: Observability for AI Agents in Production
40mKeep AI on Track: Guardrails + OpenTelemetry Observability
23mMonitoring GenAI Applications
01h 02mObserving AI Applications with OpenLit and OpenTelemetry
18mWhat did AI just do? Making LLMs Observable with OpenTelemetry