Kube-Arbitrator: A Batch System of Kubernetes

on YouTube ↗25m

" summary

Describes kube-arbitrator, an incubated Kubernetes project that adds policy-based batch scheduling features for Big Data and AI frameworks (Spark, TensorFlow) in multitenant clusters. Covers implementing advanced schedulers such as fair-share and gang scheduling on top of Kubernetes core to support resource sharing and batch job requirements.

AI-written summary · about

" chapters (5)

1. 0:00Start
2. 0:50History of kube-batch
3. 4:20Overview of kube-batch
4. 6:31Targets
5. 7:15Features

" related talks

• 24mKubernetes Batch Processing at Scale - A Scheduling PerspectiveDeliang Fan, Lim Haw Jia · Kubernetes Batch + HPC Day Europe 2023
• 33mBatch Systems in Production with Kueue: Multi-Tenancy and FungibilityAldo Culquicondor, Yuki Iwai · Kubernetes AI + HPC Day North America 2023
• 29mBuilding a Batch System for the Cloud with KueueKante Yin, Aldo Culquicondor · Kubernetes Batch + HPC Day Europe 2023
• 06mBeyond Federation: Automating Multi-cloud Workloads...Zefeng (Kevin) Wang · KubeCon + CloudNativeCon Europe 2021
• 37mIntroduction to the Kubernetes WG BatchAldo Culquicondor, Abdullah Gharaibeh, Alex Wang · KubeCon + CloudNativeCon Europe 2022
• 28mMultitenancy and Fairness at Scale with Kueue: A Case StudyAldo Culquicondor, Rajat Phull · Cloud Native + Kubernetes AI Day NA 2024
• 42mDeep Dive Into the Latest Kubernetes Scheduler FeaturesAbdullah Gharaibeh · KubeCon + CloudNativeCon North America 2019
• 25mScale your Batch / Big Data / AI Workloads Beyond the Kubernetes SchedulerAnish Asthana, Kevin Postlethwait · DevConf.US 2024