The Future of Data Science/ML at Scale
summary
Argues that the legacy platform stack fails on increasing data volume, timeliness, and quality and that brittle hand-built ETL is the bottleneck; examines the evolution of architectures and shows the lake house paradigm as the practical replacement; demonstrates how Delta Lake plus MLflow provide the pieces needed for transactional, versioned data and reproducible model lifecycle to run analytics and ML at scale, replacing fragile ETL with reproducible pipelines.
AI-written summary · about
related talks
11mRecent trends in data and machine learning technologies
50mBehind the Hype - Architecture Trends in Data
41mModern Data Architectures – What They Are and How to Implement?
41mInfrastructure for ML Applications
33mBuilding Resilient (ML) Pipelines for MLOps
50mMasterclass: Building a data stack in 2021
51mHow to Build a Modern Data Lakehouse?
34mTales of Data Architecture Evolution