Data Exploration & Experimentation with Notebooks in Azure
summary
Claim: notebooks put code, text and visuals together so non-experts can interrogate real data; example loads UK ONS Excel into a pandas DataFrame, inspects many JSON blobs exported by Application Insights from Azure Blob Storage, and finds local downloads over home broadband are slow. Moving the kernel to a cloud container (VS Code Spaces) cuts latency; scaling to a three‑node spark cluster (3 nodes × 4 CPUs, 32 GB) in Azure Synapse parallelizes processing but incurs a ~3 minute startup.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 0:35What are notebooks
- 0:50Why are notebooks important
- 2:14Example of using a notebook
- 4:03Notebook file example
- 9:20Notebook Architecture
- 17:36Azure
- 18:12Azure Storage
- 21:07Azure Blob Storage
- 24:37Hosting the Notebook
- 26:54Azure Notebooks
- 30:24Code Spaces
- 33:35Azure Synapse
- 37:35Analytics
- 45:07Summary
related talks
43mAnalyzing Big Data in Azure cloud using Spark and Databricks
44mFlying High with Data Engineering in Microsoft Fabric
24mSynapse Analytics
39mEnabling Data centric Teams
28mAccelerating Real-Time Analytics with Cosmos DB and GPU-Enhanced Serverless Apache Spark
30mProcess large-scale data and computational function in serverless manner
50mPower BI + Azure Synapse Analytics: Beginning your journey
40mData Lake House in the Cloud: The Azure Solutions