Build prod ML systems with feature /training /inference pipelines
summary
Argues monolithic ML pipelines fail and shows a Python-first refactor into feature/training/inference (FTI) decomposition with concrete code: pandas/polars/pyspark feature transforms, window aggregations, XGBoost and scikit‑learn training, and a Predictor class that loads models from a hopsworks feature store and model registry. Demonstrates point-in-time correct join to avoid leakage, batch and online APIs, saving logs, and a credit‑scoring demo running locally plus orchestration via modal and hugging face for scheduling and UI.
AI-written summary · about
chapters (10)
- 0:00Refactoring ML pipelines
- 2:17Monolithic ML pipelines fail
- 4:03FTI pattern for ML systems
- 7:12Feature pipelines in Python
- 11:45Credit scoring ML demo
- 15:40Streaming vs batch features
- 22:42Inference pipelines with stores
- 24:07Modal and Hugging Face demo
- 34:58End-to-end ML walkthrough
- 35:18Serverless ML resources
related talks
01h 12mBuild Simpler Production ML Systems using Feature/Training/Inference Pipelines
18mAutomate the boring ML stuff with pipelines
41mInfrastructure for ML Applications
38mReal Time ML Pipelines in Multi-Tenant Environments
33mBuilding Resilient (ML) Pipelines for MLOps
01h 21mBuilding a ML Model Observability pipeline
28mUnravelling Hidden Technical Debt in ML: A Pythonic Approach to Robust Systems
43mDeploying Machine Learning using sklearn pipelines