Data Science Frameworks + Managed Services
summary
Build for simplicity not novelty: a BBC mobile recommendation app takes 18 months to ship after adopting 17 components across GCP, gRPC, Kubernetes and Spinnaker; Spinnaker installs 40 subcomponents and caused more incidents than app bugs so it gets ripped out. A three‑week single Python package that writes to Elasticsearch works reliably for 12 months. JupyterHub helped data scientists ship faster. Verdict: prefer proven infra (Cosmos/Jenkins, Postgres) and avoid shiny stacks unless you can absorb their maintenance cost.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 5:30Mobile app from scratch
- 6:54Microservices mistake
- 9:02Pitfalls of gRPC
- 10:50Leaving BBC infrastructure
- 13:04Docker/Kubernetes complexity
- 15:16Spinnaker disaster
- 18:27Simplifying the stack
- 20:35Jupyter Hub workflows
- 22:01Google ML Engine
- 23:03Tech choices unknowns
- 25:32Over-engineering vs politics
- 27:17Cost of new tech
- 29:04Starting fresh
- 30:51Tech adoption advice
- 31:54Q&A
related talks
38mData Secrets from a Platform Engineer
22mBuild Smarter, Don't Just Build
38mKeeping pace with data infrastructure so it doesn't outrun you
22mProductive Machine Learning and Deep Learning Projects
21mData-driven approaches in a technology startup
28mThe Coding Portion of Data Science
50mMasterclass: Building a data stack in 2021
39mRight Code, Right Place, Right Time