Building Supercomputers Using Kubernetes and OpenHPC
summary
Defines a concrete path to run HPC workloads on Kubernetes by combining OpenHPC components and cloud-native features: uses SR-IOV and RDMA for low‑latency interconnect, exposes CPU topology via the CPU and Topology managers, mounts shared storage with Lustre, and integrates batch scheduling through an LSF connector that maps Kubernetes pods to LSF jobs. Prototype runs MPI-style jobs successfully but lacks full HPC features like job resizing and precise data‑locality; approach is feasible but not yet a drop‑in replacement for traditional supercomputers.
AI-written summary · about
chapters (23)
- 0:00Introduction
- 1:15Welcome
- 1:54Rafael Cardona Introduction
- 2:49Natale Vinto Introduction
- 3:30Agenda
- 4:45Who are you
- 7:30Supercomputers
- 15:26What is OpenHPC
- 19:06What is Kubernetes
- 20:24Components of Kubernetes
- 22:36Kubernetes Object
- 23:52Kubernetes Scheduler
- 25:23Kubernetes Scheduler Phases
- 26:14Kubernetes Scheduler with Policies
- 29:01Kubernetes and OpenHPC
- 29:54Kubernetes vs OpenHPC
- 32:20Networking in Kubernetes
- 33:53The Solution
- 36:23IBM
- 39:52Favorite projects
- 41:10Poll
- 44:06Twitter
- 44:30Conclusion
related talks
23mImplementation Challenges: From HPC to Containers in the Academy
41mBuilding & Operating a Large-scale HPC AI Cluster on Kubernetes
24mPropagating Programming Paradigms: Lifting High Performance Compute Into K8s
08mLightning Talk: Orchestrating Kubernetes Clusters on HPC Infrastructure
05mHow To Run Kubernetes Pods On My Slurm-Based HPC Center
01h 21mHPC in the cloud
19mKeynote: Building Bridges: Cloud Native and High Performance Computing
20mBenchmarking HPC workloads on OpenShift