Cloud Native + Kubernetes AI Day North America 2025
2025-11-10
Atlanta, United States
as the organisers put it
Cloud Native + Kubernetes AI Day North America 2025 has officially wrapped! Thank you to all the attendees who joined us. Watch keynotes and all breakout sessions on the CNCF YouTube Channel.
who it's for
platform engineers, site reliability engineers, ai infrastructure engineers, kubernetes operators, ml platform engineers, cloud architects, devops engineers, security engineers
topics
kubernetes-native agent runtimes; scheduler for distributed AI workloads; autoscaling GPU clusters across providers; dynamic resource allocation (DRA) for GPUs; LLM prefix aware routing; LLMInferenceService model serving; KV-cache aware autoscaling; wasm component sandboxes for MCP; observability with OpenTelemetry for AI; multinode inference and distributed training; cost-saving LLM caching (LMCache)
| title | author | length |
|---|---|---|
| Welcome + Opening Remarks | Ricardo Rocha, Rajas Kakodkar | 05m |
Opening remarks list sponsors (Cosmonic, Mirantis, Red Hat, Spectral Cloud, Suz, Lambda, Nebuse, New Forluster, Kestrel AI), logistical details for… | ||
| Beyond Chat: Bringing Agents To Kubernetes | Nimisha Mehta, Nina Polshakova | 27m |
Kagent is a Kubernetes-native, declarative agent runtime that turns CRDs into agents, model configs, and MCP server tools; it launches per-agent Deployments… | ||
| Lightning Talk: Mind the Topology: Smarter Scheduling for AI Workloads on Kubernetes | Roman Baron | 10m |
KAI is a Kubernetes-native scheduler that plans whole distributed AI workloads instead of placing pods one-by-one, claiming measurable latency and throughput… | ||
| Kubernetes Challenges at Lambda | Andrew Godwin | 06m |
Lambda runs Kubernetes twice: a platform Kubernetes on owned hardware and a managed Kubernetes for customers, and both must handle integer-percent failure… | ||
| From Open Source to Real-World Scale | Aleksandr Patrushev | 05m |
Nebius describes a vertically integrated AI cloud that promises click-to-provision access to thousands of GPUs with a claim of fully virtualized physical GPU… | ||
| Autoscaling GPU Clusters Anywhere — Hyperscalers, Neoclouds & Ba... | Lukas Gentele | 06m |
vCluster runs a full Kubernetes control plane in a single pod and integrates Karpenter to autoscale GPUs across hyperscalers, neoclouds and bare metal… | ||
| You Got a Match! LLM Prefix Aware Routing With Kubernetes | Ricardo Noriega, Cong Liu | 26m |
Kubernetes extension implements an inference gateway that routes LLM requests by initial token prefixes and maintains a global cache index to avoid GPU… | ||
| This Lying Has To Stop: Keeping AI Honest with OpenTelemetry | Whitney Lee | 25m |
An automated agent called Commit Story runs on git commit, harvesting code diffs, commit messages and every assistant chat so AI-generated journal entries… | ||
| Panel: AI Infra Best Practices: Enterprise Do’s and Don’ts | Madhuri Yechuri, Andrew Leung | 41m |
Kubernetes hosts separate clusters for batch, training, and inference at Uber, with Ray for training and Nvidia Triton for serving; GPU capacity remains scarce… | ||
| Fit-to-Serve: How a New DRA Capability for Dynamic Device... | Sunyanan Choochotkaew, Tatsuhiro Chiba | 24m |
Demonstrates a new DRA capability added to Kubernetes (enable feature gate) that exposes consumable capacity and allow multiple allocations so a single GPU can… | ||
| KServe Next: Advancing Generative AI Model Serving | Yuan Tang, Dan Sun | 24m |
KServe v0.17 introduces a dedicated LLMInferenceService CRD with model and KV caching, integration with the Envoy AI Gateway, and RMD integration for… | ||
| Wasm't That Easy: Securing MCP With Wasm Sandboxes | Jiaxiao Zhou, Taylor Thomas | 24m |
Live demos show a random MCP binary and a container both exfiltrate an SSH key when run with dangerously skip permissions, but compiling the same tool to a… | ||
| On-Call the Easy Way With Agents | Ryan Tay, Kartikeya Pharasi | 24m |
Intuit reports a deployed Agent called Mesh Debug Deputy that ties PagerDuty alerts to Splunk and Wavefront, RAGs over internal docs, and tool calls like… | ||
| LMCache: Lower LLM Performance Costs in the Enterprise | Martin Hickey, Junchen Jiang | 26m |
Claims a practical open source KV-cache layer that reduces Time to First Token and increases throughput by reusing attention tensors instead of recomputing… | ||
| The Hidden Foundation: How Effective Data Management Determines AI System Success | Keith McClellan | 26m |
Argues that production AI fails not for models but for bad data management, citing a 95% ROI-failure metric and a claim that organizations with clean data are… | ||
| Lightning Talk: My Job Says 'Running' but Nothing's Running: Kubernetes Status Reality... | Ron Kahn | 07m |
Kubernetes often reports a distributed PyTorch/MPI job as Running while no training progresses because Kubernetes only validates per-pod readiness probes and… | ||
| Lightning Talk: Resource Fairness and Utilization for Heterogeneous Batch/M... | Yuki Iwai, Gabe Saba | 10m |
Kueue (Q) implements three concrete fair-sharing mechanisms: preemption-based fair sharing that reclaims GPUs by preempting queues with higher dominant… | ||
| Cloud-Native GraphRAG: AI’s Logical Edge | Stephen Chin | 06m |
Claims that graph-augmented retrieval improves accuracy and explainability for logic-heavy tasks, demonstrated by re-evaluating a 36-seat classroom puzzle… | ||
| Closing Remarks | Yuzhui Lui, Yuan Tang | 02m |
Event wraps with explicit logistics: sponsors include platinum, diamond, and startup sponsors and attendance is strong, with organizers noting a high turnout… | ||
| nothing matches | ||