Kubernetes Batch + HPC Day Europe 2023
2023-04-18
Amsterdam, Netherlands
[Kubernetes Batch + HPC Day Europe 2022, Kubernetes Batch + HPC Day Europe 2023]
as the organisers put it
Kubernetes Batch + HPC Day has officially wrapped! Thank you to all the attendees who joined us online! Watch keynotes and breakout sessions on Kubernetes Batch + HPC Day’s YouTube channel.
who it's for
platform engineers, scheduler developers, hpc engineers, machine learning engineers, cluster operators, storage engineers, site reliability engineers
topics
fractional GPU allocations; MetaGPU device plugin; kueue job queuing layer; cloud-native job preemption; SLA-aware batch preemption; gang scheduling and DRF; data locality with Alluxio; GPU preprocessing and prefetching; kubernetes on HPC infrastructure; cluster autoscaler integration; GitOps for cluster deployment; spot instances and resource flavors
| title | author | length |
|---|---|---|
| Welcome + Opening Remarks | Ricardo Rocha | 08m |
Third Kubernetes Batch + HPC Day returns as a half‑day co‑located event, continuing after a Valencia kickoff (≈60 in person) and a full‑day Detroit edition… | ||
| Sharing is Caring – Fractional GPU Allocations With MetaGPU Device Plugin | Dmitry Kartsev | 29m |
Build that splits any GPU into 100 meta-GPUs per card, registers capacity to kubelet via a gRPC device plugin and exposes usage with mgctl and Prometheus… | ||
| Building a Batch System for the Cloud with Kueue | Aldo Culquicondor, Kante Yin | 29m |
Kueue is a cloud-native job queuing layer that integrates with the Kubernetes scheduler, job controller and cluster-autoscaler to enforce quotas, hierarchical… | ||
| Making the Most Out of Your Hardware Accelerators in a Kubernetes Cluster | Rishit Dagli, Shivay Lamba | 25m |
Naive runs with one Tesla T4 plus one vCPU are slower than two vCPUs because the GPU spends long idle time waiting on CPU preprocessing; adding prefetching and… | ||
| Kubernetes Batch Processing at Scale - A Scheduling Perspective | Lim Haw Jia, Deliang Fan | 24m |
ByteDance builds a custom Kubernetes scheduler that replaces the default to run hundreds of millions of batch pods daily and a largest cluster with >20,000… | ||
| SLA Aware Batch Scheduling in Apache YuniKorn with Multi-Tenant Preemption | Sunil Govindan, Craig Condit | 25m |
Apache YuniKorn implements SLA-aware batch preemption using hierarchical queues with guaranteed and max capacities (example: org 400MB, leaf max 200MB… | ||
| Lightning Talk: How to Bring Data Locality to I/O-Intensive Workloads on Kubernetes | Shouwei Chen | 05m |
Alluxio introduces a virtualization layer between compute and storage to serve data without copying, claiming measurable I/O gains by bringing metadata… | ||
| Lightning Talk: Orchestrating Kubernetes Clusters on HPC Infrastructure | Elia Oggian | 08m |
CSCS builds centrally managed Kubernetes across bare metal, virtual and the Alps HPE Shasta HPC system using Rancher, spins on-demand clusters via Harvester… | ||
| Lightning Talk: Does Cloud Elasticity Pay off for HPC Workloads? | Joris Cramwinckel | 10m |
Ortec Finance moves HPC from on‑prem to managed OpenShift on AWS/Azure and assembles an open‑source stack that actually works: KEDA and Knative for bursty… | ||
| Closing Remarks | Aldo Culquicondor | 05m |
This session summarizes where to contribute and which forums to join: Kubernetes Working Group - Batch is the open space for scheduler work, the Job API… | ||
| nothing matches | ||