Building Big Data Platforms: the Hidden Security Flaws You Need to Be Aware Of
summary
Claims that Azure Databricks requires egress to internet for runtimes and that DBFS can’t be technically blocked so DBFS stores production data are backed by examples of network flows, managed resource groups and runtimes fetched at cluster spin‑up; Microsoft Purview scans use Key Vault plus a self‑hosted integration runtime and still support no customer‑managed keys and high‑privilege managed identities (owner, StorageBlobDataContributor); suggested mitigations include TLS‑inspection firewalls, pre‑warmed clusters, monitoring for abnormal identity use, and automated alerts or cleanup.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 1:53Data lakehouse architecture
- 7:51Azure Databricks flaws
- 16:49Purview governance risks
- 24:21Mitigating flaws
- 28:19Risk assessment
- 30:29Q&A
related talks
51mDesigning Data Architectures that InfoSec will actually approve
48mProtecting the confidentiality and integrity of your high value data with Azure SQL and SQL Server 2
16mSecure data processing with AMD Confidential VMs on Azure Databricks
22mAzure Infrastructure - What Not to Do
31mRings of Security for Big Data in Cloud
45mKeeping Your AI/ML Data Secure- 22mAzure Infrastructure - What Not to Do
52mUncovering Azure's Silent Threats: A Journey Into Cloud Vulnerabilities