AI Pipelines With OPEA: Best Practices for Cloud Native ML Operati...
summary
Opia turns a Jupyter proof‑of‑concept RAG notebook into deployable microservices: chunking PDFs (~1,000-token chunks with overlap), embedding with a Hugging Face model, storing vectors in Chroma vector DB, retrieving top‑k chunks, reranking them, and answering with a local llama.cpp local model (4–5 GB weights). Blueprints and Docker Compose/Kubernetes helm charts handle CPU, Gaudi and GPU variants; the repo lists 20+ blueprints and ~55 contributors and the demo pipeline successfully returns RAG answers versus raw LLM guesses.
AI-written summary · about
chapters (12)
- 0:00OPEA introduction
- 1:23RAG overview and limits
- 3:09Dragon example context
- 4:59RAG notebook demo
- 11:02Notebook to deployable script
- 15:36Monolithic AI pipeline challenges
- 16:39Pipeline microservices breakdown
- 21:00OPEA standardized approach
- 22:04OPEA vision
- 26:59OPEA repository and blueprints
- 36:33Growing OPEA community
- 39:24Q&A
related talks
28mAI Pipelines With OPEA: Best Practices for Cloud Native ML Operati...
31mAIOps: Automating and Scaling ML and LLM Workloads in Production
19mScalable Data and AI Solutions with Cloud-Native Architecture
58mThe path between developing and serving machine learning models.
38mAI in Kubernetes: How to Get Started?
33mBuilding Resilient (ML) Pipelines for MLOps
29mTeam Teardown: MLOps in the Cloud for IoT
32mMLOps at Renault: A Generic Pipeline for Scalable Deployment