Common Data Science Mistakes
summary
Checklist of concrete mistakes with reproducible fixes: code and data live in R demos on GitHub and the repo is named common data science mistakes; examples use GLMs, k‑means, SVD, SHAP and even Excel. A noisy 4 outliers flips a fitted line so predicted distance at speed 25 goes from ~125 to ~60; clustering on inches vs centimeters gives different clusters until values are normalized. Covers L1/L2 regularization, data leakage, under/overfitting (polynomials up to degree 10), and bad charts like 3D pies.
AI-written summary · about
chapters (14)
- 0:00Common mistakes intro
- 2:16Domain knowledge & soft skills
- 9:44Tool selection impact
- 10:05Data prep mistakes
- 11:51Correlation vs causation
- 16:55Demo: outliers effects
- 20:09Correlated data in ML
- 24:25Sampling & leakage issues
- 25:30Normalization & clustering
- 31:08Model evaluation measures
- 33:57Model deployment & retraining
- 34:41Demo: overfitting in regression
- 38:16Visualization pitfalls
- 44:40Summary of mistakes
related talks
19mLearning from eight years of data science mistakes
34mData Science beasts (failures) and where to find them
45m"Breaking black-box machine learning"
44mKeep it Clean: Why Bad Data Ruins Projects and How to Fix it
30mData-Centric AI Cookbook-let's prep that data
22mDon’t Underestimate the Obvious: Murphy’s Law in Real-life Data Science
45mTop 10 Machine Learning Gotchas
30mBias, Explainability, and Accountability: The Data Scientist's Burden