From PoC to Viable Product: industrializing a Big Data project
summary
This talk explains how Decathlon industrializes Big Data projects, covering pipeline orchestration with Apache Airflow, continuous integration using GitHub Actions, and distributed computation on AWS EMR. It focuses on moving from proof-of-concept to a viable production product by combining these technologies and practices.
AI-written summary · about
chapters (10)
- 0:00Industrializing Big Data
- 1:47Décathlon context
- 3:53Client goals
- 5:45Data flow architecture
- 8:10Automated deployment
- 10:20Collaboration patterns
- 12:49Rapid development
- 14:14Dev environment & validation
- 16:43Future improvements
- 18:11Q&A
related talks
36mFrom POC to Prod, a data science project laid bare
39mFrom Data Science to Production - deploy, scale, enjoy!
51mIncreasing Productivity and Reducing Costs in Big Data Ecosystems on AWS
30mDevOps For Data Teams
35mScaling up to Big Data Devops for Data Science
34mData Products, From 0 to data science pipeline
19mDevDataOps - Data-Driven Product Development for Software Engineers
01mintro video