Right Code, Right Place, Right Time
summary
Argues that most data science problems are actually software engineering problems and that success turns on concrete choices: dependency management, reproducible bootstrapping, and deployment pipelines. Describes using Docker images, shell scripts to bootstrap EC2 and occasional AWS spot instance jobs, S3 data snapshots, simple dashboards and scheduled jobs (Airflow-style) and warns many homemade solutions become Rube Goldberg technical debt rather than scalable production systems needing a machine learning engineer class of infrastructure.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 3:19Type B data science
- 6:57Data science reality
- 12:00Engineering challenges
- 20:10Why these challenges
- 23:05Solving these problems
- 28:30Engineering solutions needed
- 31:31ML technical debt
- 34:08DevOps in data science
- 36:47Why ML harder than mobile
- 38:10Future of data science roles
- 38:52Q&A
related talks
28mThe Coding Portion of Data Science
41mBetter Code for Data Science
38mKeeping pace with data infrastructure so it doesn't outrun you
38mSimplicity: The Key to Software Success
35mApplied Data Science
44mClean Machine Learning Code: Practical Software Engineering...
26mSoftware Delivery from First Principles
42mHow to be a 10x Data Scientist