WHAT DOES MLOps LOOK LIKE FOR LARGE LANGUAGE MODELS (LLMs) IN PRODUCTION?
summary
Demonstrates an Atlassian Virtual Assistant for JSM implemented as a two-tier system combining an in-house intent classifier plus a ChatGPT integration that pulls Confluence articles, uses embeddings and vector database retrieval, and composes prompts into the model’s context window. Argues industry practice favors in‑context learning and small specialized models over fine‑tuning, warns of model drift and observability gaps (research shows GPT‑3.5/4 performance drift), notes hallucination risks so the intent classifier is kept, and flags cost scale — an estimate suggests ~$36M annual hit for search at Google scale using GPT‑3.5.
AI-written summary · about
chapters (12)
- 0:00MLOps for LLMs intro
- 2:49LLMs vs traditional ML
- 3:32Atlassian virtual assistant
- 9:16MLOps vs LLMOps
- 11:43Task alignment for LLMs
- 13:28Retrieval and computation
- 14:53In-context learning
- 18:47Model serving challenges
- 20:54LLM reliability risks
- 21:59Model observability
- 23:03LLM cost estimation
- 25:29Takeaways and future
related talks
49mLLMops: what's beyond ChatGPT and how you can deploy it
47mOperationalization of AI/ML models and use cases for ChatGPT Q&A
52mUnleash the Power of Large Language Models (LLMs)
30mLarge Language Models and Prompt Engineering for Developers
43mBeyond the Hype: A Realistic Look at Large Language Models
49mA Practical Approach to Using LLMs: Peculiarities and Pitfalls
33mOpening Keynote: LLMs, between fantasy and reality - the poin...
51mTowards a Pragmatic Understanding of Large Language Models