Using Azure DevOps for Azure Data Factory
summary
Four concrete deployment patterns for Azure Data Factory: single Data Factory (dev and prod same resource), ADF_publish branch which contains the ARM template for factory.json that Azure Data Factory emits when you hit publish and suits Azure DevOps release pipelines, per-component JSONs deployed with PowerShell cmdlets (requires handling dependencies and removals), and a wrapped approach using Kamil’s PowerShell module / Azure DevOps marketplace task that automates dependency ordering and gives granular control.
AI-written summary · about
chapters (21)
- 0:00Introduction
- 0:40Agenda
- 1:17Download slides
- 1:39Azure Data Factory
- 1:54What is your Data Factory
- 2:58Data Factory Components
- 4:49Data Factory DevOps
- 5:43Connect Data Factory
- 7:05Publish
- 8:18Deploy
- 11:07Getting the source code
- 11:41Exporting an ADF template
- 13:35Deploy Data Factory
- 13:45Build Data Factory
- 14:04Branching
- 14:15Release Pipeline
- 15:15Gateways
- 15:50Arm Template
- 16:35Arm Template Option 3
- 18:47Arm Template Option 4
- 19:57Summary
related talks
01h 07mApplication Lifecycle Management in Azure Data Factory
46mAutomating Infrastructure as Code with Azure DevOps
56mDataDevOps for the Modern Data Warehouse on Microsoft Azure
31mAzure DevOps for System Engineers
41mA Case Study - Data Movement and Transformation using Azure Data Factory
01h 05mWorkflows of Highly Functional App & Data Engineering Teams
01h 00mBuilding cloud-native data pipelines using Azure Data Factory
20mChoosing between Microsoft Fabric, Azure Synapse Analytics and Azure Data Factory