Kubernetes Batch + HPC Day North America 2022
2022-10-24
Detroit, United States
as the organisers put it
Bringing together a community of experts who want to make Kubernetes the best tool to manage infrastructure for research, training, and scientific use cases
who it's for
platform engineers, hpc engineers, scheduler developers, research computing engineers, cloud architects, site reliability engineers, devops engineers
topics
Kubernetes Job API extensions; indexed Jobs and suspended jobs; CPU manager control plane; multi-NIC high bandwidth networking; MPI Operator and large-rank MPI; Flux/Fluence hierarchical scheduler integration; armada multi-cluster batch scheduler; hybrid cloud bursting for OPC; large-scale autoscaling to 1k-10k pods; checkpointing and core pinning techniques; CNCF batch working group coordination; device plugins and accelerator support
| title | author | length |
|---|---|---|
| Welcome + Opening Remarks | Ricardo Rocha | 08m |
Opening remarks summarize that this is the second Kubernetes Batch + HPC Day with a full day of one keynote, eight sessions and four lightning talks after a… | ||
| Keynote: Fair Share - What Shared Responsibility Means for Managed Kubernetes Cluste... | Mickey Boxell | 08m |
Managed Kubernetes shifts specific responsibilities: providers operate kube-apiserver, kube-controller-manager, kube-scheduler, etcd while users typically keep… | ||
| Propagating Programming Paradigms: Lifting High Performance Compute Into K8s | Marlow Weston | 24m |
Argues Kubernetes keeps useful pieces like device plugins and pluggable infrastructure but lacks HPC guarantees: requests node and network behavior consistency… | ||
| Hybrid Cloud Bursting Electronic Design Analysis Optical Proximity C.... | Derren Dunn, Gaurav Singh | 35m |
Hybrid cloud bursting of Optical Proximity Correction (OPC) runs on managed OpenShift (ROSA) uses commercial tool Calibre by Siemens, centralized license… | ||
| Lightning Talk: Fluence: Approaching a Converged Computing Environm... | Daniel Milroy, Claudia Misale | 15m |
Describes a concrete stack: improved MPI Operator plus a Scheduling Framework plugin originally called KubeFlux now Fluence, integrating the Flux hierarchical… | ||
| Lightning Talk: Evolving the Job API to Become Defacto Standard for Batch Worklo... | Aldo Culquicondor | 29m |
Kubernetes Job API adds concrete features: indexed Jobs give each pod a numeric index accessible via env var and scale to 5,000+ pods; suspended jobs let… | ||
| Lightning Talk: CNCF Batch Working Group Update | Alex Scammon | 13m |
CNCF Batch Working Group organizes end-user focused discussion of cloud‑native batch scheduling across Volcano, Armada, MCAD, YuniKorn, Slurm, HTCondor and… | ||
| 79 Lightning Talk Dancing with Cores A Path For FineGrained Core Positioning within Kubernetes | Ata | 13m |
A CPU control plane is under development to mix shared and pinned cores and later add isolated cores, with CPU control plane exposed via cluster primitives… | ||
| Make Kubernetes Networking Ready for World Class AI and HPC ... | Sunyanan Choochotkaew, Gaurav Singh | 27m |
Kubernetes gets multi-NIC support via an operator that automates interface discovery, host route setup, IPAM and syncs state so pods get direct high-speed… | ||
| Building Armada – Running Batch Jobs at Massive Scale on Kubernetes | Jamie Poole | 35m |
G-Research describes Armada, a batch scheduler that actually runs millions of jobs per day across many clusters (tens of thousands of nodes, thousands of… | ||
| Panel Discussion: Fragmentation of the Batch Ecosystem in Kubernetes, Challenges and Solutions | Abdullah Gharaibeh, Diana J. Arroyo, Wilfred Spiegelenburg, Daniel Milroy, Albin S | 53m |
Panelists compare concrete systems and limits: Armada, Unicorn, MCAD and Flux (qflux/fluence) build schedulers, queuing, quotas and hierarchical borrowing… | ||
| Closing Remarks | Abdullah Gharaibeh | 01m |
A call for action creates ongoing coordination: a Kubernetes Batch Working Group forms as a year‑round forum, aligned with the CNCF Batch System Initiative and… | ||
| Coordinate Workloads Colocation: QoS-Oriented Scheduling Enhancement on K8s | Zuowei Zhang, Tao Li | 32m |
Kubernetes add-on Koordinator implements QoS-aware colocation by adding sub-QoS classes, a colocation profile webhook, a code-manager recommender for resource… | ||
| Managed Kubernetes — Next Gen Academic Infrastructure? | Viktória Spišaková, Lukáš Hejtmánek | 31m |
Managed Kubernetes runs a production academic platform with 2500 CPU cores, 60–100 TB flash, 50 GPUs supporting ~200 users and ~1,000 pods/day, integrated with… | ||
| Beyond Experimental: Spark on Kubernetes | Weiwei Yang | 30m |
Builds a cloud‑native Spark platform that hides multiple Kubernetes clusters behind a batch processing gateway which accepts job specs and maps virtual queues… | ||
| nothing matches | ||