Designing Data Architectures that InfoSec will actually approve
summary
Reference architecture uses Azure Data Factory, Databricks, Azure Data Lake and Azure Synapse wired into a hub‑and‑spoke VNet with private endpoints and v‑net injection for Databricks; secure cluster connectivity and managed VNets remove public egress but require planning IP space. KeyVault, RBAC and managed identities handle secrets; audit/log analytics and Azure Sentinel cover monitoring. Pitfalls: CI/CD breaks if resource locks are enabled, soft‑delete on Delta lakes balloons storage, and a locked‑down dev environment tripled costs in one case.
AI-written summary · about
chapters (12)
- 0:00Intro
- 0:41THE DATA PLATFORM
- 3:55WHAT'S THE IMPACT
- 6:40THE FOUNDATIONS
- 8:30PRIVATE ENDPOINTS
- 10:53DATABRICKS VNET PEERING
- 11:53DATABRICKS VNET INJECTION
- 14:51DATA FACTORY ON-PREMISES CONNECTIVITY
- 23:10WHAT ABOUT AZURE SYNAPSE?
- 25:49SUMMARY
- 32:48SECURITY FEATURES
- 38:07USE WITH CAUTION
related talks
31mBuilding Big Data Platforms: the Hidden Security Flaws You Need to Be Aware Of
46mThe Ultimate Jigsaw - Architecting Azure Synapse Analytics
51mIntroduction to Azure Data Platform
42mDesigning Enterprise Scale Cloud Platform on Azure
24mREDLake Azure - Enterprise Data Lake and Business Satellites
40mData Lake House in the Cloud: The Azure Solutions
16mSecure data processing with AMD Confidential VMs on Azure Databricks
37mModern Architecture in the Cloud of 2026