DevOps for DS: Experiences from building a data science platform
summary
Covers building a cloud-based data science platform to simplify productionizing machine learning and data science applications by applying cloud infrastructure and appropriate DevOps abstractions. Describes practical experiences and lessons from creating that platform to streamline building and deploying models and data-driven apps.
AI-written summary · about
chapters (7)
- 0:00DevOps for Data Science Experiences from building a cloud-based data science platform
- 3:03Managing Data Science in Production is Hard! The tools and practices are not very mature . Everyone ends up buiding their own solutions • Building own solutions require careful system architecture and
- 4:10The Goal: The Effective Data Science Team The data science team is self-sufficient to build end-to-end ML applications
- 13:58Runtime Base software setup for the project.
- 20:03Behind the Scenes + Two docker images are built for each project-One for CPU and another for GPU
- 35:02Observations Running a Python function as an API is hard! doesn't have to be!!
- 36:41What about ML models? Write your predict function and run it as API.
related talks
30mDevOps For Data Teams
29mDevOps for Data Science? - automate the boring stuff and leverage the OSS ecosystem
59mDevOps for Data Science
26mApplying DevOps practices in Data and ML Engineering
20mDevDataOps - Using data for exponential growth of your organization
56mDevOps for Machine Learning
30mA Devops Approach to Data: A Tale Through the Trenches in Search of a Cloud Data Revolution
43mDevOps in Cloud