HPC in the cloud
summary
Claim: modern cloud stacks can run true HPC-style distributed training using existing MPI tooling and container schedulers, not just vague “examples.” The session shows concrete tools — MPI and torchrun, PyTorch, Volcano and MiniKube for local gang scheduling, AWS Batch and Metaflow for cloud runs — cites 175 billion parameter models and a reported 7500-node Kubernetes cluster, demos a multi-node job using 16 GPUs (2×8) and shows logs and loss curves so the build actually runs.
AI-written summary · about
chapters (8)
- 0:00HPC in the cloud intro
- 1:46AI and HPC convergence
- 10:53Distributed AI training
- 22:44HPC in AI workflows
- 31:52Demo: AI jobs on Kubernetes
- 52:11Orchestrating AI systems
- 1:00:04Demo: Scaling AI with Metaflow
- 1:13:43Summary and resources
related talks
09m⚡ Lightning Talk: Advanced GPU-Orchestrated Workflows and HPC Integrations on K8s for Distributed AI/ML at Scale
36mAn almost perfect match: cloud native and scientific computing
47mCloud Native and Sustainable, Reproducible Scientific Computing
19mKeynote: Building Bridges: Cloud Native and High Performance Computing
46mMachine Learning with Kubernetes
35mState of Containers and the Convergence of Hyperscale, BigData and HPC
27mFrom Supercomputing to Serving: A Case Study Delivering Cloud Native Foundation Models
05mSponsored Keynote: Towards Clouds of AI Clusters