Keep AI on Track: Guardrails + OpenTelemetry Observability
summary
Shows a concrete pipeline that instruments guardrails across UI, orchestration, LLM, retrieval and infra using OpenTelemetry and an LGTM backend (Loki/Tempo/Mimir + Grafana) to capture traces, logs and metrics. Demonstrates a dockerized React frontend + Python backend trip‑planner where OpenLit enforces and emits guardrail spans and attributes, compares NVIDIA NeMo and LangChain tradeoffs, and records semantic conventions (about 15–20 gen_ai fields). Tracks actionable metrics like pass/fail rates, top violation categories, guardrail-induced latency and token costs so teams can alert, audit and reduce silent failures.
AI-written summary · about
chapters (7)
- 0:00Enterprise AI guardrails intro
- 4:39Guardrails in production
- 9:55OpenTelemetry for guardrails
- 15:30AI frameworks comparison
- 17:17Guardrail observability demo
- 28:15Production guardrail systems
- 31:06Conclusion & Q&A
related talks
29mElevating generative AI and LLM observability
01h 02mObserving AI Applications with OpenLit and OpenTelemetry
40mOptimizing AI Applications for Production with Observability
16mWhat Did the AI Just Do?: Making LLMs Observable with Open Telemetry
26mSponsored Session: Navigating Open Source AI with Guardrails
18mWhat did AI just do? Making LLMs Observable with OpenTelemetry
30mFrom Black Box To Insight: Observability for AI Agents in Production
33mWhat Did My Agent Do? Observability and Accountability for AI Agents