How to Build a Data Platform from Scratch
summary
How to build a data platform from scratch: explains why a company needs centralized data collection (for BI, machine learning integration, or customer data delivery) and describes the components and steps to create a data platform. Covers practical architecture and engineering choices for gathering, storing, processing, and providing data for analytics and services.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 3:53Data ingestion and storage
- 8:13Spark data processing
- 11:05Data transformation and aggregation
- 13:33Task schedulers and orchestration
- 15:44OLAP and analytics
- 19:22ML content moderation
- 21:34Customer communication
- 22:16Bot mitigation
- 23:43Recommendation service
- 28:01BI and dashboards
- 34:43Final architecture
- 37:35Key takeaways
- 40:55Q&A
related talks
23mHow to Build a Data-Intelligence Startup within your Organization
33mAnalytics Platform at Petabyte Scale
16mBuilding a Computing Environment to Support Analytics
01h 05mHow to build your data engineering/data science team?
21mHow Not to Build a Data Platform Lessons from Failure to Success
30mBuilding the best data team is not rocket science
35mBuilding a data analytics platform on strong opinions held loosely
07mWhere to start your Data Analytics Journey