Harnessing generative AI with NVIDIA AI and Microsoft Azure
summary
Demo values: NVIDIA H100 GPUs run as Azure ND H100 v5 VMs with 8 GPUs per VM, 3.6 TB/s bisectional intra-node bandwidth and 450 GB/s all-reduce, scaling across 400 Gb links; MIG can split an H100 into 7 instances. Benchmarks show ~7x training speed on BERT and >4x inference throughput with 99.9% accuracy using the Transformer Engine. Platform pieces: NVIDIA AI Enterprise software on Azure ML, Nemo foundation models plus Picasso and BIO Nemo, and H100 hardware-backed confidential computing.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 0:30What is generative AI
- 2:00Enabling generative AI
- 2:58Challenges with generative AI
- 4:38Foundation models
- 6:46What does it take
- 7:41NVIDIA H100
- 9:21Matt Vegas Azure
- 11:59Microsoft Azure
- 14:02Nvidia AI Enterprise
- 16:27Confidential Computing
- 18:40Performance
- 21:33Nemo Framework
- 24:18Questions
related talks
52mSimplifying the adoption of generative AI for enterprises
22mQuickly deploy NVIDIA optimized generative AI on Azure AI Studio
50mAccelerate development and deployment of Gen AI with NVIDIA on Azure
31mPanel: The Future of AI
22mAI @Scale
28mScaling agentic AI: From infrastructure to enterprise outcomes
39mDisrupt and Grow with Microsoft Azure AI
25mOptimize productivity by meeting next-level demands of the AI era