Implement a Serverless Data Analytics strategy with Microsoft Azure

on YouTube ↗47m

" summary

Generates fake ad reports (clicks, conversions, impressions) and ingests them into ADLS Gen2 with hierarchical namespace using append blobs and date partitioning, writes Parquet, then queries and joins files with a Synapse serverless SQL pool and Spark. Uses Data Factory copy activity (parallel) to consolidate into three Parquet outputs and Data Flows or Spark for schema drift and aggregation. Demonstration has occasional glitches but consolidation, queries and a Delta Lake updatable Parquet workflow complete successfully.

AI-written summary · about

" chapters (9)

1. 0:00Introduction
2. 3:02Data governance fundamentals
3. 8:26Data mesh principles
4. 10:33Data ingestion strategies
5. 18:23Azure Synapse Analytics
6. 24:13Consolidating datasets
7. 35:01Merging schemas
8. 42:14Delta Lake updates
9. 47:09Q&A

" related talks

• 52mImplement a Serverless Data Analytics strategyRoberto Freato · Big Data Conference Europe 2022
• 43mImplement a Serverless Data Platform with Microsoft AzureRoberto Freato · Big Data Conference Europe 2023
• 49mAnalytics in SQL Server and Azure SQL Databasechuck heinzelman, Ajay Jagannathan · SQLBits 2023
• 48mBuilding an Azure Data Analytics Platform End-to-EndPaul Andrew · SQLBits 2023
• 30mProcess large-scale data and computational function in serverless mannerReenu Saluja · AzConf 2020
• 52mPower BI + Azure Synapse Analytics. All you need to start creating a unified analytics solutionPawel Potasinski · SQLBits 2023
• 45mBuilding a Lakehouse on the Microsoft Intelligent Data PlatformSimon Whiteley, Craig Porteous · SQLBits 2023
• 32mSynapse Serverless SQL Pool: SQL without databaseLjubica Vujovic Boskovic · Heapcon 2022