Policy-based Compute Orchestration for AI/ML Workloads
summary
Shows a working pattern that glues Metaflow job submission to policy-driven placement with Kyverno + Metaflow, using an external configmap-driven routing approach to mutate node affinity and enforce validate policies. Examples include routing GPU requests to a node group advertised as 16 vCPU / 64GB, 8 CPU / 32GB, 8 GPUs, denying unblessed images with a deny policy, and preventing single workloads from reserving entire clusters. It succeeds at guardrails but does not replace autoscalers; preemption can waste 7-hour runs and needs checkpointing.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 3:54Why policies for Kubernetes
- 6:02AI/ML workload challenges
- 9:39Metaflow for AI/ML
- 13:10Policy guardrails for AI/ML
- 17:06Kerno’s compute policies
- 23:13Metaflow and Kerno combined
- 24:17Challenges and trade-offs
- 26:47Conflicts with Karpenter
- 31:40Q&A
related talks
40mOptimizing AI/ML Workloads
32mHuman-Friendly, Production-Ready Data Science Stack With Metaflow & Kubernetes
32mOptimizing Machine Learning Training Infrastructure: A Governance Approach
39mScalable Multi-Node AI Workloads in Multi-Tenant AI Clouds U...
35mBuilding a Medical AI with Kubernetes and Kubeflow
35mBuilding Robust Reactive ML Systems In A Multiplayer Setting
22mBeyond Single-Cluster Limits: Scaling GPU Workloads Across Kubernetes With…
36mHuman-Friendly, Production-Ready Data Science Stack with Metaflow & Kubernetes