Big Data Moonshots and Ground Control
summary
Argues that 80% of analytic effort is data cleaning and that automation alone fails, offering a human-in-the-loop system called predictive interaction that watches live visuals of raw rows, predicts transformations, shows previews, and appends chosen steps into a reusable script. Demonstrates anomaly detection, example-based pattern learning that converges with few examples, interactive joins, and the ability to export to Hadoop so the recorded pipeline runs at scale. The demo shows practical reduction in manual cleanup time.
AI-written summary · about
chapters (6)
- 0:00The need for ground control
- 1:07Mythology of data heroes
- 2:45Designing for the user
- 4:49Core design principles
- 6:38User personas
- 7:23Predictive interaction demo
related talks
20mSkrub DataOps, Probabl
34mPast The Hype: A Better Way To Think About Big Data
21mSmarter, faster Data analytics with generative AI and Machine Learning
30mDevOps For Data Teams
48mBecoming a Data Head, a reflection on the trade offs and decision
25mHow AI Transforms Enterprise Data Pipelines with 85% Gains
45mWhat’s Next in AI for Data and Data Management
09mAutomating cognitive tasks to increase business value