F3: A Compiler For Feature Engineering
summary
F3 is a Python DSL and compiler that converts feature pipelines into a canonical semantic form so the system can prove equivalence, move filters safely across left joins, and detect repeated computation for reuse; it implements normalization by evaluation into a normal form split into columns and constraints, and adds intersection types plus bounded quantification to propagate semantic labels across expressions. Deployed at scale, it finds unused tables, deduplicates work across tens of thousands of features, and yields query and storage savings.
AI-written summary · about
chapters (14)
- 0:00F3 compiler intro
- 1:30Dependent types foundation
- 4:20Feature engineering importance
- 6:23Feature engineering in SQL
- 9:33Compiler challenges
- 11:21Functional programming solution
- 12:25Normal forms for validation
- 16:24Data lineage with normal forms
- 17:47Type system extensions
- 22:06Conclusion and future
- 22:47Q&A: F3 features
- 25:41Q&A: Temporal features
- 26:44Q&A: Benefits and performance
- 29:13Q&A: Query optimization
related talks
22mA Type System For Feature Engineering (Extended Abstract)
38mFeatran, Featran77 - Generic Feature Transformer for Data Pipelines
45mAnatomy of a production ML feature engineering platform
36mDeclarative Feature Engineering
13mDesigning Feature DSLs: Principles and Tradeoffs
28mData VS Engineers: a long lasting story
20mUsing Flink for Versatile Feature Engineering Pipelines
01h 28mFeature Engineering Made Simple