Deploying Machine Learning using sklearn pipelines
summary
Sklearn pipeline objects provide a reproducible, modular workflow that encodes feature extraction, imputation, transformers, ensembles and grid search into one serializable object. A concrete example uses the Titanic dataset to show a median-age bug that disappears once the training median is fixed inside the pipeline, proving identical transforms on train and test. Custom DataFrameUnion and an XGBoost wrapper let non-sklearn models act as steps. Pipelines dump to disk and load for prediction so deployment is a matter of jo. dump / jo. load not retraining.
AI-written summary · about
chapters (15)
- 0:00Deploying ML with sklearn pipelines
- 2:07What are sklearn pipelines
- 8:25Pipeline advantages
- 10:12Transformers and estimators deep dive
- 18:42Building basic pipelines
- 19:04Data science lifecycle with pipelines
- 25:23Model selection and ensembles
- 26:49Hyperparameter optimization
- 29:16Publishing and deploying models
- 31:01Deployment mistakes to avoid
- 33:48Data scientist-engineer collaboration
- 35:13Pipeline deployment benefits
- 37:18Pipeline community benefits
- 37:38Key takeaways
- 38:05Q&A
related talks
18mAutomate the boring ML stuff with pipelines
35mPandas, Pipelines, and Custom Transformers
36mBuild prod ML systems with feature /training /inference pipelines
33mUsing Scikit-Learn's interface for turning Spaghetti Data Science into Maintainable Software
35mMachine Learning Pipeline using Luigi and Scikit Learn
39mA full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and cons
32mLarge-scale machine learning pipelines using Luigi,...n
05mLightning Talk: Build a Reproducible ML Workflow with Kubeflow Pipelines