Full-stack Machine Learning and Generative AI for Data Scientists
summary
Shows concrete pipeline pieces: uses metaflow to write versioned flows, decorates steps to run on Kubernetes and deploys with Argo workflows, indexes docs into pinecone and queries via OpenAI for RAG. Demonstrates a random-forest model on over a million data points, visualizes cards, freezes conda environments, and wires markdown chunker → postprocessor → vector indexer so runs trigger one another. Code and commands for creating, triggering and scheduling flows appear throughout.
AI-written summary · about
chapters (12)
- 0:00Full-stack ML intro
- 1:04What is full-stack ML
- 6:42Does ML change stack
- 12:00ML infrastructure stack
- 18:45Prototype to production
- 23:50Setting up the sandbox environment
- 35:55Production ML workflows
- 46:04Answering audience questions on Metaflow
- 57:54Full-stack GenAI intro
- 1:07:46RAG systems with Metaflow
- 1:20:16Deploy RAG workflows
- 1:27:12Summary
related talks
36mHuman-Friendly, Production-Ready Data Science Stack with Metaflow & Kubernetes
32mHuman-Friendly, Production-Ready Data Science Stack With Metaflow & Kubernetes
01h 03mOne Flow to Rule Them All: Building the end to end ML project to production
35mBuilding Robust Reactive ML Systems In A Multiplayer Setting
22mGenerative AI Model Data Pre-Training on Kubernetes: A Use Case S...
43mMachine Learning Engineering principles with Python and MLFlow
34mData Products, From 0 to data science pipeline
31mMLflow: Platform for Complete Machine Learning Lifecycle