Writing a scikit-learn compatible estimator in the modern age
summary
Live code builds a dummy classifier, a stateless transformer and a meta "timer" wrapper and makes each pass scikit‑learn tests by using check_estimator, cloning subestimators, delegating methods and tags, and calling validate_data and check_is_fitted. Key gotchas: fitted attributes must end with an underscore, fit must return self, classifiers must set classes_ and newer APIs set n_features_in_; implement or delegate get_feature_names_out and use available_if for dunder behavior. Persistence recommendation: use scops.io for safe serialization.
AI-written summary · about
chapters (10)
- 0:00Introduction to custom estimators
- 3:53Why write your own estimator
- 14:33Building a custom classifier
- 18:12Creating a custom transformer
- 22:07Developing a meta estimator
- 30:34Using estimators in pipelines
- 34:07Grid search with custom estimators
- 37:01Conventions and best practices
- 39:30Future developments in scikit-learn
- 40:12Q&A and serialization
related talks
33mUsing Scikit-Learn's interface for turning Spaghetti Data Science into Maintainable Software
01h 26mSktime—Python Toolbox for Time Series: How to Implement Your Own Estimator
27mPyTesting the Limits of Machine Learning
29mGetting Scikit-Learn To Run On Top Of Pandas
01h 24mInpsect and try to interpret your scikit-learn machine-learning models
28mGet the best from your scikit-learn classifier
42mCreating correct and capable classifiers
32mExtending Spark Machine Learning Beyond Linear Regression