Accelerate your ML innovation with AWS ML infrastructure services
summary
AWS reports that P4d A100 instances cut training cost ~60% vs prior GPUs and deliver ~3x throughput; a T5 run on 2,000 A100s drops from ~12+ days to under 6 days (≈2x). Scale features include UltraClusters with thousands (up to ~4,000) GPUs, 400Gb networking, EFA and GPU Direct plus FSx storage. For inference, Inf1 Inferentia shows up to 70% lower cost; Alexa moves ~80% of speech synth to Inf1, saving ~30% cost and reducing latency ~25%.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:17ML adoption
- 2:28AWS ML stack
- 9:35P4D instances
- 11:25Ultra clusters
- 17:49GPU solutions
- 19:53GPU families
- 22:49Trainingbased instances
- 24:54Inference
- 26:33Inferential Instance
- 28:34Code Example
- 31:28Getting started
- 32:36Summary
- 33:57Outro
related talks
34mHPC on AWS: Innovation without infrastructure constraints
31mRealize price-performance advantage for AI
09mDemocratizing Access to GPUs for Faster MLAI Innovation with Generative AI
30mAccelerate deep learning & Habana Gaudi–based Amazon EC2 DL1 instances
44mScaling ML from 0 to millions of users
24mGenerative AI Chips
50mThe democratization of tomorrow’s technology
35mMLDS 2019: Madhusudan Shekar of Amazon at Machine Learning Developers Summit 2019