Introducing AIBrix: Cost-Effective and Scalable Kubernetes Control Plane for VLLM
Talkat KubeCon + CloudNativeCon China 2025 · on Tuesday, 2025-06-10· at 16:15· Level 19 | Crystal Court I·
summary
AIBrix is a Kubernetes-native control plane built to scale VLLM-based large language model inference by providing a pluggable architecture for LLM-specific autoscaling, high-density LoRA management, distributed KV cache, heterogeneous serving, and model loading. It focuses on deep co-design with inference engines (like vLLM) to enable advanced optimizations and improved resource utilization, and the talk includes demos and benchmarks of these components.
AI-written summary · about
related talks
34mImprove AI Inference (serving models) With KServe and VLLM
28mChildren's Guide To LLMs on Kubernetes
28mAnyscale's Unified Platform for LLM Development and Deployment
14mAI Deployment: Mastering LLMs with KFServing in Kubernetes
07mTidal Auto-scaling for Training and Inference Based on Kubernetes + KEDA
32mAI Beyond Autocomplete: Using LLMs To Create 1000 Kubernetes Controllers
12mVertical Mobility: Inference from MVP to Trillion Parameters
42mStop Treating LLMs Like REST APIs