Scalable Feature Engineering with Hamilton
summary
An open source framework called Hamilton converts each feature into a single Python function, decouples execution via a driver, and adds runtime data validation and auto-generated docs so transforms become unit-testable and inspectable. At Stitch Fix it manages 4,000+ features and speeds monthly model/feature updates four times faster. Scaling delegates compute to Ray/Dask/Spark integrations, so the thing built runs in production and parallelizes without changing feature logic.
AI-written summary · about
chapters (10)
- 0:00Hamilton introduction
- 1:27Hamilton’s origin story
- 8:05Simplifying feature engineering
- 19:01Hamilton in ML workflows
- 20:45Scaling Hamilton for teams
- 24:14Runtime data validation
- 26:20Hamilton’s readability
- 27:02Scaling compute with Ray
- 31:35Open source progress
- 35:04Q&A
related talks
21mHamilton: an open source framework for feature transform code in Python
36mDeclarative Feature Engineering
32mA feature Store for Machine Learning
45mAnatomy of a production ML feature engineering platform
33mEvolution and Scaling of Feature Store at Uber
01h 28mFeature Engineering Made Simple
30mHow Feature Stores Enable Operational ML
40mEnabling simple ML ops at Stitch Fix