Building the Bridge Between DevOps and MLOps in a GenAI world
summary
Argues companies almost never train full LLMs (costs in the hundreds of millions) and instead should self-host open models when data must stay inside the firewall. Shows a working RAG demo built with Olama + Llama 3.1, Postgres + PGVector, and lmer index on a GPU node, and confirms it returns accurate answers. Recommends version everything (prompts, models, RAG data in YAML manifests), use evals for automated tests, and productionize via containerization, Kubernetes and multi‑node HA.
AI-written summary · about
chapters (11)
- 0:00Introduction and speaker background
- 3:18MLOps family tree and GenAI shift
- 6:52LLM Ops vs classical MLOps
- 10:00Consumption models for LLMs
- 12:11DevOps manifesto for MLOps and LLM Ops
- 17:06Quick demo setup and tools
- 21:27Building a RAG pipeline demo
- 23:14AI spec and version-controlled manifests
- 25:26Productionizing the demo
- 27:12Versioning and testing LLMs
- 28:18Q&A and resources
related talks
01h 02mWorkshop: GenOps: Building a MLOps Platform to Support GenAI Workloads with Open...
31mAIOps: Automating and Scaling ML and LLM Workloads in Production
41mTHE STATE OF MLOPS - MACHINE LEARNING IN PRODUCTION AT ENTERPRISE SCALE
53mExperience with an end-to-end MLOps pipeline: Practical insights and 3D architectural perspectives
33mBuilding Resilient (ML) Pipelines for MLOps
31mMLOps In Practice – How To Run Your Machine Learning Models In Production At Enterprise Scale
51mSpeeding up deployment of machine learning models
56mDevOps for Machine Learning