Cloud Native AI + Kubeflow Day 2026
2026-03-23
Amsterdam, Netherlands
as the organisers put it
Cloud Native AI + Kubeflow Day Europe 2026 has officially wrapped! Thank you to all the attendees who joined us. Watch keynotes and all breakout sessions on the CNCF YouTube Channel.
who it's for
ml platform engineers, platform engineers, kubernetes operators, machine learning engineers, site reliability engineers, data engineers, infrastructure architects
topics
federated LLM inference; Cilium Cluster Mesh networking; multi-cluster orchestration (Nova); vLLM serving and autoscaling; GPU reservations and scheduling; bare metal Kubernetes AI factory; ModelPack/ModelKit model packaging; Envoy AI Gateway routing; AI API gateways and semantic routers; observability for AI data pipelines; Slurm over Kubernetes (Soperator); NVIDIA GB200 optimization
| title | author | length |
|---|---|---|
| Welcome + Opening Remarks | Yuan Tang, Rajas Kakodkar | 07m |
Welcome to the combined Cloud Native AI + Kubeflow Day with opening remarks that list concrete next steps: Kubernetes AI conformance specs 133–135 and a coming… | ||
| Federated llm-d: Elevating Distributed Inference Beyond Clus... | Madhuri Yechuri, Abhishek Malvankar | 26m |
Federated llm-d stretches LLMD across clusters using Cilium Cluster Mesh for cross‑cluster networking, Nova as the multi‑cluster orchestrator, body‑based… | ||
| Sponsored Keynote: Building Your AI Factory: Wh... | Andreea Munteanu | 06m |
Argues that scalable AI at enterprise requires an AI factory built on bare metal Kubernetes with explicit GPU schedulers (mentions Volcano or Kai) to manage… | ||
| Sponsored Keynote: The 2026 Infrastructure, Opera... | Kevin Cochrane | 08m |
Vultr positions itself as an alternative hyperscaler offering a full AI stack with 33 data center regions, support for Nvidia and AMD GPUs, and claims to serve… | ||
| Sponsored Keynote: Production AI Runs on Open Sou... | Gleb Evstropov | 09m |
Nebius runs an AI platform built on Kubernetes, Slurm, Ray, PyTorch and DeepSpeed that combines cloud‑native orchestration with HPC batch scheduling to support… | ||
| Ethical Bias | Robert Glenn, Diana Zhawn | 19m |
Claims that fully ethical AI is impossible now but practical progress comes from design choices: a communication-focused agent prototype learns from users… | ||
| The Sanity Check: Observability for AI Data Pipelines | David Aronchick | 27m |
Claim: attach provenance at ingestion so a bad LLM answer can be traced to the exact source. Shows an open spec called data bill of materials and an OSS… | ||
| Expl(AI)n Like I’m 5: An Introduction To AI-Native Networking | Niranjan Shankar, Ram Vennam | 32m |
Kubernetes can host GenAI but requires new networking primitives: training uses RDMA/Infiniband and dynamic resource allocation for GPUs; serving needs… | ||
| Smarter Databases in Kubernetes: Applying AI for Query Optimization and Self-Tuning | Peter Zaitsev | 35m |
Argues that Kubernetes is a natural control plane for adaptive databases and that current systems sit around level two autonomy, not fully autonomous, with a… | ||
| Panel: Routing Intelligence Vs Traffic Control: Arc... | Abdullah G, Morgan F, Eitan Y, Nili G, Dan Sun | 41m |
Gateway Inference Extension (GAIE) debates whether routing logic lives out-of-band as an external processing call or in-path as a next-hop Service. Practical… | ||
| Rollout on Reception: Progressive Delivery for GenAI Apps Using Real... | Thomas Vitale, Whitney Lee | 24m |
Live demo shows users scanning a QR, voting thumbs up/down every 30 seconds and Knative Services + Flagger adjusting canary weights (start 20% → 40% → 50%) in… | ||
| Baby Sitting Your AI Models in Production? STOP! | Ram Iyengar | 18m |
Claims a reproducible, low-touch MLOps workflow using the CNCF sandbox ModelPack and its directory spec ModelKit that bundles code, model weights, notebooks… | ||
| Kubeflow Trainer Observability: Real-Time Progress Tracking for Reliab... | Abhijeet Dhumal, Rob Bell | 29m |
Adds real-time training metrics into TrainJob status on the Kubernetes API so Katib can prune trials and save GPU-hours instead of parsing logs or running… | ||
| GPU Reservations: Maximizing Utilization and Fairness Across Teams | Sam Huang, Thomas Chaton | 28m |
Company pair builds a Kubernetes platform that combines reserved capacity with autoscaling to claim up to 75% cost savings while keeping researcher elasticity… | ||
| Sign Language Recognition Improvements Since the London Keynote | Rob Koch | 27m |
Prototype moves from static snapshots to dynamic gesture tracking with 21 hand keypoints (X/Y/Z) fed through MediaPipe into ML models for fast handshape… | ||
| Real-Time GPU Job Scheduling Latency Prediction in Multi-Cluster Kubernetes | Sujoy Dutta | 22m |
Bloomberg runs a Karmada controller that computes live GPU queue ETAs by tracking scheduler objects and measuring the delta from creation to last-transition… | ||
| Running ML Challenges on Kubernetes (Part 2) | Hannes Hansen, Paulo Guilherme Pinheiro Pereira | 19m |
CERN runs a production prototype that turns Codabench into a Kubernetes-native challenge platform: a Helm chart is merged upstream and the executor now creates… | ||
| From Hallucinations To Hardware: Diagnosing LLM Failures | Ryan Peirce | 23m |
Live demo shows an LLM app that surfaces both entertaining hallucinations and real GPU faults and traces them from user request to infrastructure using… | ||
| Kubernetes Meets Climate Data: Designing Analysis-Ready Dat... | Armagan Karatosun, Anna-Lena Erdmann | 22m |
Built a Kubeflow-based pipeline that uses Kubeflow pipelines, DuckDB and Xarray/Zarr to turn heterogeneous EO products into analysis-ready cubes with… | ||
| A Guide To AI API Gateways and Semantic Routers | Max Korbacher | 25m |
Compares concrete gateways and routers: Envoy AI Gateway with its two-tier edge + inference design for routing between cloud providers and on‑prem models… | ||
| When Models Write Malware: Defending Against On-demand Exploits | Shane Lawrence | 26m |
Live demo shows a Claude-generated exploit beating VirusTotal signature scanners and landing on a toy vulnerable plugin endpoint, then Falco eBPF rules and… | ||
| A Deep Dive on How To Leverage the NVIDIA GB200 for Ultra-Fast... | Kevin Klues, Jan-Philip Gehrcke | 28m |
NVIDIA GB200 NVL72 hardware connects 72 GPUs across 18 nodes with NVLink at ~800 GB/s and the DRA-based ComputeDomain CRD exposes IMX-level partitions to… | ||
| Scaling LLMs at Bloomberg Using Envoy AI Gateway: A Hybrid Cloud Approach | Xiaolin Lin, Aaron Choo | 31m |
Bloomberg converts Envoy and Envoy Gateway into a production GenAI control plane called Envoy AI Gateway that front-ends AWS/GCP/Azure and on‑prem models… | ||
| Lightning Talk: AI Meets Kubernetes: Langflow & GROQ in Automated Incident Management | Harini Anand | 13m |
Proof-of-concept pipeline automates detection, diagnosis, and safe remediation of recurring Kubernetes incidents by combining LangFlow visual orchestration… | ||
| Lightning Talk: Roche's Kubeflow Story: Large Models in Seconds, Costs Cut, Securit... | Oswaldo Gomez | 11m |
Roche runs Kubeflow endpoints that saw 10–15 minute cold starts for 13–26 GB GPU images; adopting Karpenter EC2 node class with prebuilt EBS snapshot plus… | ||
| Lightning Talk: Intelligent Traffic Routing for Distributed LLM Inference: Beyond Trad... | Zhonghu Xu | 11m |
Kthena Router is a production-grade, Volcano-integrated router that replaces per-model gateways (10 models no longer means 10 gateways) and natively implements… | ||
| Lightning Talk: K8s Issue #52757: Sharing GPUs Among Multiple Containers | Xiao Zhang | 10m |
Kubernetes GPU sharing remains an open engineering problem since 2017 and now sees a practical stack: Dynamic Resource Allocation (DRA) for scheduler… | ||
| nothing matches | ||