Observing AI: Why Even LLMs Sometimes Need Therapy
summary
Covers how to implement observability for large language models (LLMs), focusing on practical monitoring techniques, anomaly detection, and load testing for generative AI. Describes ways to inspect internal model behavior and presents real-world applications and tools to apply these observability practices in daily workflows.
AI-written summary · about
chapters (8)
- 0:00AI observability
- 1:04LLM challenges
- 3:13OpenLIT and OpenTelemetry
- 6:21Demo
- 8:49Advanced monitoring
- 9:50Cost metrics
- 10:32Helm deployment
- 10:53Next steps
related talks
06mObservability for Large Language Models
18mAI Observability: How To Fix Issues With Your ML Model
29mElevating generative AI and LLM observability
41mGenerative AI: bliss or miss?
40mMeasuring the unmeasurable: How to evaluate generative AI-based systems?
16mWhat Did the AI Just Do?: Making LLMs Observable with Open Telemetry
32mEnd-to-End Observability for LLM Inference: From Token To GPU
14mImproving Large Language Model Performance