Data Transformation: A Framework for Exploratory Data Analysis
summary
Introduces a repeatable exploratory framework and applies it to the EPA vehicle fuel economy dataset, showing concrete steps: identify entities and hierarchies, create quintile bins, run clustering (start ~8 clusters, reduce to 4), make pivot heatmaps, and build graph transformations with community detection using Pandas Scikit-Learn Seaborn and NetworkX. Produces network visualizations and time series of connections, claims about about one insight per minute, and launches an open source project cultivar that codifies parts of the workflow.
AI-written summary · about
chapters (12)
- 0:00Data transformation & EDA intro
- 4:17Exploring the data framework
- 8:54Prep: data types & entities
- 13:07Adding analysis categories
- 16:44Explore: filtering & aggregation
- 19:14Pivot tables for multi-variable
- 21:20Vehicle data over time
- 22:48Scatter matrices for fields
- 24:34Graph analysis for entities
- 28:24Framework review
- 29:46Key takeaways & future
- 31:12Closing & resources
related talks
38mTransforming Data to Unlock Its Latent Value
01h 51mData Visualization and Exploration with Python
32mEDA Techniques for Data Analysis
24mThe TL;DR of EDA
25mPandas on Jupyter: Improving Your Data Analysis Workflow
30mBe kind to yourself! Spend (less) time on Data Exploration
01h 28mPython Visualization for Exploration of Data
01h 38mEffective Data Visualization