Building Data-Intensive Teams
summary
Project recommends a workflow: validate hypotheses with a fast SQL prototype, collect real user feedback via controlled A/B tests, then harden infrastructure and scale using event pipelines like Kafka, Avro, Parquet. Cites concrete signals: 350 million monthly users at the marketplace, Facebook data showing ~80% of AI inference cycles go to recommendations, and a Bing UI tweak that adds >$100M/year. Emphasizes build-minimum-to-learn, avoid premature ML platforms or feature stores, and design services to survive a 10x growth.
AI-written summary · about
chapters (10)
- 0:00Introduction to data intensive teams
- 2:57History of data applications
- 8:37Modern data-driven products
- 11:27Costs and challenges of data apps
- 15:59From idea to experiment
- 19:08Prototyping and feedback loops
- 21:35Balancing speed and stability
- 23:41Building scalable systems
- 27:15Scaling and reinvention
- 28:43Q&A on measurement and feedback
related talks
35mHigh-performance data foundation for AI success
49mPanel discussion - Building Data Products
23mBig data and big insights
28mAccelerating AI at Scale with Data
22mBuild Smarter, Don't Just Build
45mChris Harbert: Testing AI at the Scale of Big Data
14mBuilding a Data-Driven Culture: An Anthology of Strategies and Insights
19mScaling & Industrialization of AI