ML inference acceleration for lightweight VMMs
summary
vAccel is a lightweight framework and paravirtual runtime (vAccelRT) that exposes ML and linear-algebra acceleration operators to VM tenants without embedding full vendor runtimes in functions. The implementation provides modular backends that bind to acceleration stacks (jetson-inference/TensorRT, Google Coral TPU, with pathways for CUDA and OpenCL) and is integrated into KVM VMMs QEMU and AWS Firecracker to enable inference acceleration for lightweight VMs on x86_64 and aarch64. A demo uses containerized deployment to show vAccel in action.
AI-written summary · about
related talks
39mAccelerating Open-Source RL and Agentic Inference with vLLM
24mOptimizing CPU LLM Inference in PyTorch: Lessons From VLLM
41mBoost ML on Heterogeneous AI Accelerator with Ray on K8s
17mEmpower ML Framework on RISC-V by Backend Graph Compilers
16mAccelerate HPC and AI Applications with RVV Auto-Vectorization
23mHugging Face Accelerate: Making Device-Agnostic ML Training and Inference Easy...
32mEmpower WebAssembly ML Over Diverse AI Hardware Accelerators Outside the Browser
29mMulti-Accelerator PyTorch Serving With NxD Inference and vLLM