How Amazon built Nova FMs using Amazon SageMaker
summary
Amazon builds the Nova family of foundation models on Amazon SageMaker HyperPod using thousands to tens of thousands of accelerators (NVIDIA H100 P4D/P5 and AWS Trainium), FSx for Lustre + S3, EFA networking, EKS scheduling, Airflow and MLflow for orchestration, and PyTorch/NVIDIA NeMo. They combine tensor parallelism, pipeline and data parallelism, plus quantization and distillation to cut inference cost, and HyperPod’s fault‑replacement and checkpointing keeps months‑long runs viable; Perplexity reports a 40% reduction in training time.
AI-written summary · about
chapters (9)
- 0:00Introduction and welcome
- 1:51Amazon Nova foundation models
- 7:52Challenges in training foundation models
- 12:24Technology stack for Nova models
- 15:33Memory and compute walls
- 22:35Handling failures at scale
- 25:23SageMaker HyperPod solution
- 28:51HyperPod features and benefits
- 29:51Key takeaways and next steps
related talks
19mDeveloping large-scale machine learning algorithms on Amazon SageMaker
27mFoundation Models: Blueprints for success
39mTrain ML models at scale with Amazon SageMaker Training
25mAccelerate NLP training with Amazon SageMaker
19mUsing Amazon SageMaker to develop and deploy ML models
35mAccelerate your ML innovation with AWS ML infrastructure services
25mSponsored Session: Accelerating GenAI Inference: From AWS Deep Learning...
21mAWS On Air ft. Amazon SageMaker Inference Recommender