Feature Engineering Made Simple
summary
Claims a reproducible, procedural feature pipeline built as Anovos that runs on Spark and demos on a Kaggle credit‑risk set (with six simulated time periods). It surfaces a feature recommender + mapping, applies automated quality checks, five imputation methods, variable clustering and WOE encoding, and a data stability metric that predicts drift. It rejects blind auto‑feature generators, reports which features drift, and shows mean time‑to‑update improving from ~17 to ~45 days in their deployments.
AI-written summary · about
chapters (36)
- 0:00Start
- 0:27Feature Engineering Made Simple
- 2:58Voice over
- 7:31Why Why Do We Need Feature Engineering
- 7:55What Is Feature Engineering
- 10:17Candidate Features
- 23:13Enable Scalable Feature Engineering
- 26:17Ingest and Analyze Phase
- 31:09Measure of Central Tendency
- 31:48Measures of Percentiles
- 33:02Histograms for Numerical and Categorical Attributes
- 35:05Biasedness Detection
- 37:16Attribute Associations
- 37:40Correlation Matrix
- 40:01Variable Clustering
- 41:23Drift and Stability
- 43:47Historical Data
- 46:52Data Stability Analysis
- 49:22Attribute Stability
- 56:11Outlier Detection
- 56:37The Quality Checker Module for Auto Detection
- 57:33Distribution Using Measures of Percentile
- 1:01:23Invalid Entry Detection
- 1:03:47Feature Selection
- 1:04:31Cold Start Problem of Feature Engineering
- 1:10:07Feature Mapper
- 1:11:41Output of the Mapper
- 1:14:20The Expression Parser
- 1:15:39Log Transformations
- 1:16:02Three Ways To Generate Candidate Features
- 1:18:08Stability
- 1:21:45Null Columns Detection
- 1:22:27Auto Imputation
- 1:23:13Categorical Attributes Encoding
- 1:24:59Variable Clusterization
- 1:25:55Latent Features
related talks
36mDeclarative Feature Engineering
02h 17mFeature Engineering is for Everyone!
39mFeature Engineering: Unlocking Potential for Predictive Models
35mTime-series Feature Engineering Done Right
45mAnatomy of a production ML feature engineering platform
21mTime for quality ML from event-based data
17mCreating & operating ML models from event-based data using feature stor...
30mFeature Engineering: An Apprentice’s Guide to the “Dark Art” of Machine Learning