Keynote: Building a Large Model Inference Platform for Heterogeneous Chinese Chips Based on VLLM
Talkat KubeCon + CloudNativeCon China 2025 · on Wednesday, 2025-06-11· at 10:00· Level 16 | Grand Ballroom I·
summary
Explains adapting the open-source vLLM inference engine to run on domestic Chinese GPUs by adding support for chip-specific acceleration features (PageAttention, Continuous Batching, Chunked Prefill), performing performance bottleneck analysis, and developing custom chip operators. Describes deploying the adapted vLLM on Kubernetes using the open-source llmaz project, including heterogeneous GPU scheduling, monitoring, and elastic scaling practices.
AI-written summary · about
related talks
08mKeynote: Plug in and Scale: Serving LLM Models on Kubernetes Ma...
39mAccelerating Open-Source RL and Agentic Inference with vLLM
31mThe Evolution of Multi-GPU Inference in vLLM
10mKeynote: Scaling Open Source Large Language Models
17mWaferLLM: Large Language Model Inference at Wafer Scale
12mKeynote: How LLMs Have Changed the Way Organizations Do AI
36mCloud-Native Model Serving: vLLM's Lifecycle in Kubernetes
22mIntroduction to Large Language Models (LLM) on Kubernetes