Cloud Native + Kubernetes AI Day NA 2024
2024-11-12
Salt Lake City, United States
as the organisers put it
Cloud Native + Kubernetes AI Day North America has officially wrapped! Thank you to all the attendees who joined us. Watch keynotes and all breakout sessions on Cloud Native + Kubernetes AI Day’s…
who it's for
platform engineers, machine learning engineers, site reliability engineers, mlops engineers, kubernetes operators, cloud architects, data engineers, security engineers
topics
kuberay multi-cluster scheduling; topology-aware GPU scheduling; kueue fair sharing and preemption; llm agents on kubernetes; retrieval-augmented generation pipelines; kubeflow with lakeFS reproducibility; model serving with KServe and Model Mesh; multicloud burstable inference; multi-nic portable network CRs; prompt injection and ai supply-chain attacks; incremental GPU slicing; charm++ and mpi runtimes on kubernetes
| title | author | length |
|---|---|---|
| Welcome + Opening... | Amber Graner, Rajas Kakodkar, Ricardo Rocha | 06m |
Welcome and logistics note lists sponsors GMI and red hat, announces captioning and a QR code for the code of conduct, and says the program team selects from a… | ||
| SkyRay: Seamlessly Extending KubeRay to Multi-Cluster Multi-Cloud Operation | Anne Marie Holler | 24m |
Extends KubeRay to schedule Ray clusters, jobs and services across multiple Kubernetes clusters and clouds by interoperating with a policy-driven fleet… | ||
| Attack, Defense & Danger in the Age of AI | Shane Lawrence | 25m |
Live demo runs a Kubernetes workload called Whopper GPT that scales a tic‑tac‑toe deployment to 50 and uses a discovered system prompt token to control… | ||
| Advancing Cloud Native AI Innovation Through Open Collaboration | Yuan Tang | 05m |
Red Hat positions its cloud native AI stack as 100% open source, building Red Hat OpenShift AI and Red Hat Enterprise Linux AI from upstream tools while… | ||
| Multitenancy and Fairness at Scale with Kueue: A Case Study | Aldo Culquicondor, Rajat Phull | 28m |
Kueue augments Kubernetes scheduler and ClusterAutoscaler to run batch/ML workloads with fair sharing (v0.7) and reclaimable quotas via preemption, claiming… | ||
| LLM Powered Agents with Kubernetes | Hema Veeradhi, Shrey Anand | 25m |
Shows a working LLM agent stack deployed on OpenShift Kubernetes that routes queries to department-specific RAG tools (product, HR, accounts) and returns… | ||
| From Supercomputing to Serving: A Case Study Delivering Cloud Native Foundation Models | Autumn Moulder | 27m |
A small team scales from one cloud to clusters across four cloud providers in under six months by building cross‑cloud plumbing that bursts stateless inference… | ||
| Charm++ on Kubernetes Cloud | Aditya Bhosale | 08m |
Charm++ runtime runs on Kubernetes by reusing the MPI operator with minimal code changes and adding an MPI operator CRD changes that introduce min/max replicas… | ||
| Dressing-up Your Cluster for AI in Minutes with a Portable Network CR | Sunyanan Choochotkaew, Tatsuhiro Chiba | 24m |
Claim: a single portable Network CR automates per-node secondary NIC discovery, IP range allocation and VPC registration so clusters get pod-to-pod… | ||
| Boosting Training and Inference Performance via Topology-Aware Scheduling of Heterogeneous Resources | He Cao | 20m |
ByteDance claims catalist raises LLM training and inference throughput by adding topology-aware scheduling on Kubernetes: it implements die-level affinity and… | ||
| Brag Your RAG with the MLOPS Swag | Madhav Sathe, Jitender Kumar | 30m |
Live demo builds three RAG paths: LangChain + Pinecone, Canopy + Pinecone, and a pure LangChain variant without a vector DB; all deploy on Kubernetes and the… | ||
| Reproducible AI with Kubeflow, lakeFS and Langchain | Oz Katz | 26m |
Builds a RAG chatbot from Project Gutenberg texts stored on S3, vectorizes chunks into a local vector DB, and queries OpenAI for answers while demonstrating a… | ||
| Inference on Streaming Data at Scale at Intuit | Sri Harsha Yayi, Vigith Maurice | 24m |
Numaflow is a Kubernetes-native open-source platform that claims to let ML teams plug Python/Java/Go/Rust UDFs into pipelines that read from Kafka, Pulsar, SQS… | ||
| Incremental GPU Slicing in Action | Abhishek Malvankar, Olivier Tardieu | 27m |
An open‑source controller called insta-slice operator incrementally creates and destroys NVIDIA MIG slices without kernel or device‑plugin API changes by… | ||
| Transform Your Kubernetes Cluster Into a GenAI Platform: Get Ready-t... | Kenji Kaneda | 09m |
LLMariner turns a Kubernetes cluster into a GenAI platform by providing an OpenAI-compatible API for inference, embeddings, fine-tuning and RAG while… | ||
| Cost Saving Strategies for Interactive AI Development | Shravan Achar | 08m |
Uses Apache YuniKorn to enforce fair sharing, user-specific policies and budget limits while carving an interactive compute pool (eg, 100 cores reserved when… | ||
| Closing Remarks | Amber Graner, Rajas Kakodkar, Ricardo Rocha | 01m |
Organizers thank sponsors G, sponsor s, and go sponsor, acknowledge speakers who applied and came to present, and note the reception is starting now; attendees… | ||
| The Evolution of MLOps | Alex Yeh | 05m |
GMI Cloud describes itself as a GPU Cloud for AI workloads and claims improving GPU reliability compared with recent Llama 3 reports while expecting rapid… | ||
| nothing matches | ||