⚡ Eliminating Adaptation Latency: Day-0 Inference for New Models on Multi-Backends With...
Talkat KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China 2026 · 2026-09-07..09-09·
summary
Explains how vLLM enables Day‑0 inference for new models on heterogeneous hardware (specifically Ascend NPUs) by using a pluggable, out‑of‑tree architecture to avoid blocking mainline development, Custom OP mechanisms for non‑invasive operator extensions, and main2main synchronization plus CI tooling to keep Ascend backends aligned with vLLM releases. Focuses on reducing per‑model porting time from weeks to hours through modular design and reproducible integration practices.
AI-written summary · about
related talks
39mAccelerating Open-Source RL and Agentic Inference with vLLM
31mThe Evolution of Multi-GPU Inference in vLLM
11mKeynote: Building a Large Model Inference Platform for Heterogeneous Chinese Chips Based on VLLM
29mMulti-Accelerator PyTorch Serving With NxD Inference and vLLM
42mStop Treating LLMs Like REST APIs
24mCost and Performance Optimization of LLM Inferencing.
10mLightning Talk: Pluggable PyTorch LLM Inference Architecture With VLL...
21mScaling LLM Inference with vLLM