Elevating generative AI and LLM observability
summary
Generative AI adoption requires end-to-end observability: Amazon Bedrock for models and guardrails, Dynatrace AI/LLM Observability for token, cost and trace data, and trace loop (OpenTelemetry) to instrument pipelines. A demo shows a personalized travel app using RAG with cache fallbacks, Bedrock calls, and guardrails with traceEnabled set; token counts, model-by-model token breakdown, and guardrail hit rates surface issues. Cited numbers: ~50% of AI use cases need customization and 41% of orgs struggle to define ROI; events are retained (default 30 days, extendable to years) for auditability.
AI-written summary · about
chapters (8)
- 0:00Introduction and acknowledgments
- 1:13Speaker introductions
- 5:08GenAI trends and challenges
- 10:25AWS generative AI stack
- 13:55Amazon Bedrock capabilities
- 18:36Observability in GenAI applications
- 21:05Demo walkthrough
- 27:21Open source resources and closing
related talks
22mObservability in AI-Powered Applications with Amazon Bedrock
40mKeep AI on Track: Guardrails + OpenTelemetry Observability
41mGenerative AI: bliss or miss?
33mWhere Are We Going With Observability?
28mWhere are we going with observability?
38mGenerative AI - Architectures and applications in depth
32mGenerative AI for Observability: Automating Root Cause Analysis in Modern IT Systems
12mObserving AI: Why Even LLMs Sometimes Need Therapy