Zipline - A Declarative Feature Engineering Library
summary
Zipline is Airbnb’s declarative feature library that claims to cut feature engineering from months to days by letting users declare features in config and generating point-in-time correct features for both offline training and online inference. It runs inside the Bighead platform, fuses temporal joins with aggregations using a binary roll-up tree that tiles windows so non-group aggregates scan O(log n) nodes instead of millions, handles deletes via reversible group transforms, and uses a lambda architecture with snapshots and binlogs for correctness.
AI-written summary · about
chapters (12)
- 0:00Introduction to Zipline
- 1:35Feature engineering challenges
- 5:31Time-series joins in ML
- 10:47Aggregation properties
- 13:33Sliding window optimizations
- 17:04Change data handling
- 19:55Query topology design
- 22:03Feature serving stack
- 24:55Data layout in storage
- 26:44Data architecture overview
- 29:11Declarative API design
- 31:18Talk summary and Q&A
related talks
26mIntroducing Zipline: An Open Source Feature Engineering Platform
36mReal-Time ML Pipelines: Feature Chaining with Chronon
36mDeclarative Feature Engineering
20mUsing Flink for Versatile Feature Engineering Pipelines
45mBuilding a Feature Platform to Scale Machine Learning
01h 28mFeature Engineering Made Simple
32mA feature Store for Machine Learning
20mStreaming Feature Extraction for a ML Pipeline using Memphis.dev and Bytewax.