Optimizing AI/ML Workloads
summary
Claims a concrete, deployed system: Metaflow for transparent usage reporting and cost attribution that instruments Kubernetes/Argo runs, captures node labels and pod metrics into VictoriaMetrics, and attributes spend to flows and teams. Shows surprises like 500% memory demand on nodes, a single $6,000 Deepseek training flow, and 20% price differences between GPU instance choices. Describes usable features—retry and checkpoint to avoid rerun cost, dev vs production pools, monitoring agents—and gives actionable knobs that actually reduce waste.
AI-written summary · about
chapters (16)
- 0:00Optimizing AI ML workloads
- 1:49Cost versus value
- 5:02Metaflow for AI workflows
- 8:43GPU workloads optimization
- 10:30Predicting AI resource needs
- 12:58Maximizing real-time utilization
- 15:49Resource contention in clusters
- 18:41Dev and prod compute pools
- 20:03Minimizing rerun costs
- 21:46Choosing GPU instances
- 23:33Speed versus cost trade-offs
- 24:15Cost attribution tracking
- 26:20Monitoring for cost attribution
- 28:23Q&A on model transition
- 31:58Q&A on monitoring priorities
- 33:45Q&A on AI spending trends
related talks
19mResource Management Innovation for AI Workloads in Cloud Computing
35mResource Monitoring and Optimization with Metaflow
36mHuman-Friendly, Production-Ready Data Science Stack with Metaflow & Kubernetes
26mKubernetes and MLOps for Scalable and Reproducible Generative AI
35mBuilding a Medical AI with Kubernetes and Kubeflow
27mHarnessing Kubernetes for Scalable AI/ML Workloads
27mApplying AI/ML: How to Reduce Effort, Time and Cost
27mGreening the AI Cloud: Validating Power Models for Kubernetes Containers