Exploratory Data Analysis: The Missing Step for AI
summary
Argues that skipping exploratory data analysis causes AI failures and shows a concrete EDA workflow using R programming and open libraries to inspect data quality, engineer features, run NLP bigram checks, and perform Spearman correlations. Demonstrates real failures (Amazon hiring bias, redlining‑like location model), recommends at least 1000s of datapoints, and shows a built model where referring domains and URL length actually predict article traffic. Describes when to stop, what to drop, and when data is unusable.
AI-written summary · about
chapters (9)
- 0:00Introduction to AI and EDA
- 3:18Why AI projects fail
- 7:48Exploratory data analysis lifecycle
- 13:46Initial data quality checks
- 18:00Requirements verification
- 21:50Feature engineering process
- 25:44Building effective AI teams
- 30:36Ethical AI and future risks
- 34:32Key takeaway: Ask what could go wrong
related talks
32mEDA Techniques for Data Analysis
39mDoing Data: The Critical Process of Data Preparation (...)
26mHow to develop useful and unbiased AI
26mThe Hidden Foundation: How Effective Data Management Determines AI System Success
30mData Prep Ops: The Missing Piece of the MLOps Puzzle
21mIs It All About the Data?
11mData-centric AI: A new paradigm
30mData-Centric AI Cookbook-let's prep that data