DataEngBytes 2023 - BRI-T1-06 - Yanir Seroussi
summary
Argues that data engineering often outweighs data science in market value — job ads show about 3× as many data engineering roles at $100–150k and 4× at $200k — and gives hands-on examples: backing a recommendation service on mongodb caused database-level locks and later replacing the approach with an event logger to S3 (Kafka/Kinesis principles); building a small Python ML pipeline that stays in production; discovering an A/B test bug that invalidates six months of data. Key takeaway: pick the right tool, verify data, and build pragmatic pipelines that actually run.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 1:47Five lessons from data science
- 2:31Shiny tech isn’t always shiny
- 7:07Principles matter more than tools
- 10:40Solve problems, not titles
- 14:16Full stack skills drive impact
- 18:33Focus on what matters
- 20:40Key takeaways and Q&A
related talks
40mData Engineering in a Rapidly Changing World
51mPanel Debate: Rise Of The Data Engineer
25m"Data Engineering: The World Between Worlds"
28mSoftware Engineering Skills for Data Engineers
31mThe State of Data Engineering
38mThe Rise & Downfall of the Data Engineer REVISITED
31mFuture of data engineering: what's in store?
50mThe future of Data Engineering panel