Data cleaning: when less is more
summary
Covers strategies for variable selection and building data-cleaning pipelines to reduce high-dimensional feature sets for ML. Describes univariate variable selection, PCA (Principal Component Analysis), and penalized regression techniques as concrete methods, and offers practical tips for combining them to avoid expensive grid-search-based selection.
AI-written summary · about
chapters (13)
- 0:00Start
- 0:55Inspiration for this talk
- 2:53Why do we need to clean data at all?
- 5:37Understanding terms
- 7:17Univariate feature selections
- 10:14Amount of variation
- 12:22Pairwise feature selection
- 13:32Impute missing values
- 15:49Imputė missing values
- 18:32Pairwise correlation
- 22:22Principal component analysis
- 29:50Regularization techniques
- 30:14Penalized regression
related talks
40mA Practical Guide to Dimensionality Reduction Techniques
34mDimensionally reducing data squeezing out the good stuff
49mData Cleansing & Beyond: How to Address Data Debt for AI
38mWhy giving your algorithm ALL THE FEATURES does not always work
48mThe Missing Step in Moving from Data to Models
28mAI/ML for Non Data Scientists
39mDoing Data: The Critical Process of Data Preparation (...)
30mFeature Engineering: An Apprentice’s Guide to the “Dark Art” of Machine Learning