Keynote: How We Save $900 per Day with Self-Hosted AI: Building Scalable Local LLM Infrastructure
Talkat KubeCon + CloudNativeCon China 2025 · on Wednesday, 2025-06-11· at 09:48· Level 16 | Grand Ballroom I·
summary
Describes how an e-commerce site replaced SaaS LLMs (e.g., OpenAI) with a self‑hosted stack on private cloud GPUs to cut costs ~90% and retain control over customization, privacy, and throughput. Covers the orchestration technologies used to run inference workloads on cloud GPUs and practical strategies for building stable, scalable, high‑performance local LLM applications.
AI-written summary · about
related talks
34mKeynote: Building for AI at Scale: Infra, Integrity, & Innovation
20mSecure, private LLMs in your cloud with Anyscale Endpoints
16mKeynote: An Open-Source Stack for AI Compute: Kubernetes + Ray + PyTorch + vLLM
12mKeynote: AI Infrastructure
16mKeynote: Accelerating AI/ML Development with Open Source Software
36mKeynote: AI, ML, Cloud for ML
11mKeynote: Open Source Stack Empowers the AI Era
20mSelf-hosting AI LLMs - a beginners guide