Lightning Talk: LLM-Driven Applications: Intro to LLM Agents in Kubernetes
Lightningat AppDeveloperCon NA 2024 · on Tuesday, 2024-11-12· at 17:00· Salt Palace | Level 1 | 151 G·
summary
Shows an LLM agent running on Kubernetes that auto-detects new tools and agents and coordinates them to complete multi-step tasks. Demonstrates OpenAI-style function calling where the model chooses which API-wrapped tools to invoke, using a resource usage retriever plus a resource cost calculator called eight times (2 resources × 4 months) to get an exact billing estimate. Recommends mapping existing endpoints into tool descriptions, using serverless for infrequently invoked tools, and iterating prompts and parameters.
AI-written summary · about
chapters (7)
- 0:00LLM agents in Kubernetes
- 1:02Read vs write LLM interactions
- 3:08Limitations of retrieval augmented generation
- 4:54LLM agent actions
- 6:39Tool calling and APIs
- 8:03Designing LLM apps
- 9:27Demo: LLM agent
related talks
25mLLM Powered Agents with Kubernetes
10mLightning Talk: Hybrid LLM Inference With LangChain and Agent2Agent
42mLLMs on Autopilot: Running AI Agents on Kubernetes With Open Source Tools
30mLLMs on Autopilot: Running AI Agents on Kubernetes with Open Source Tools
09mLightning Talk: Sailing Kubernetes Operation with AI Power
14mSponsored Session: Lightning Talk: Efficient Inference Serving with Kubernetes Gateway...
08mKeynote: Plug in and Scale: Serving LLM Models on Kubernetes Ma...
23mAgents in Kubernetes 101: A Crash Course on Cloud Native AI Agents