From SSIS to Synapse to Fabric - A Data Architects Case Study
summary
A metadata-driven declarative framework migrates an EDW automation system from SSIS to Synapse/ADF and then Fabric, keeping the same metadata-driven declarative framework and swapping payload templates (DTS/SSIS packages -> copy into/OpenRowset/SQL procs -> Spark notebooks). A controller/worker orchestration pattern runs extracts, transforms and loads; it supports tens of enterprise warehouses (10–20) but trades raw latency (milliseconds on-prem -> seconds/minutes in cloud) for cloud integration and cheaper DW run costs, and uses Visual Studio + REST API unit tests to validate pipelines.
AI-written summary · about
chapters (11)
- 0:00Introduction to SSIS to Fabric
- 2:11The era of imperative SSIS
- 4:41Moving to declarative frameworks
- 9:05Metadata framework demonstration
- 15:37Migrating to Synapse Analytics
- 18:59Synapse implementation demo
- 24:34Synapse lessons and performance
- 32:11Adopting Microsoft Fabric
- 35:45Fabric implementation demonstration
- 40:33Future features and wish list
- 45:26Unit testing and conclusion
related talks
50mMicrosoft Fabric Data Factory Q&A
41mFinOps, how data engineers get their cloud cost under control
27mModernize your data integration to enable petabyte scale analytics with Microsoft Fabric
50mWhen and how to migrate to Azure Synapse?
31mData Mesh & Data Vault
53mAzure Synapse Analytics: Evolving Azure SQL Data Warehouse
46mMicrosoft Fabric Synapse data warehouse Q&A
19mAccelerate your data potential with Fabric