What is the MVP for Data Engineering projects?
summary
A cautionary playbook: an attempted end-to-end MVP that wires a rules engine into the mobile front end fails because the first use case (an engineer-made anniversary pop-up) is contrived and not useful; a second project integrates a new feature store and picks two representative features, but ingesting data for millions of users bogs pipelines down until the team decides to make big data small by sampling users, pruning columns and accepting ~1 million rows as enough to hit production and iterate.
AI-written summary · about
chapters (3)
related talks
35mBuilding Data-Intensive Teams
25m"Data Engineering: The World Between Worlds"
30mValue Engineering: How to Get Your Big Data Investment.....
32mAn Only One Step Ahead Guide for Machine Learning Projects
26mData & Analytics Strategy: 5 Overlooked Components
24mDataEngBytes 2023 - MEL-T2-02 - Vikas Rajput
31mBuilding a Data Experimentation Platform Without a Platform Team
26mApplying DevOps practices in Data and ML Engineering