Self-Hosted LLMs: A Practical Guide
summary
Guide shows a full, working local setup: download a quantized Whisper model (~466 MB from Hugging Face), build a C++ whisper.cpp binary into a container with Podman, run the image and expose the service on port 8001, and call it from a Streamlit UI to transcribe English and French audio. Demonstration proves a laptop-scale self-hosted pipeline that reduces API costs, keeps data private, and serves useful ASR results in real time.
AI-written summary · about
chapters (9)
- 0:00Introduction to LLMs
- 1:04Defining large language models
- 2:13Understanding LLM technology
- 5:44Interaction levels with LLMs
- 12:12LLM application lifecycle
- 14:48Benefits of self-hosting
- 18:06Containerized deployment
- 21:04Whisper demo and workflow
- 29:28Future outlook and Q&A
related talks
45mEverything You Need to Know About Running LLMs Locally
20mSelf-hosting AI LLMs - a beginners guide
34mPrivacy First: Building LLM-Powered Web Apps with client side WASM
47mOwn Your LLM: Running Generative and Agentic AI Locally
40mOpen Source LLMs in the Cloud: Scalable Solutions
43mLarge Language Models for Devs: from zero to your first LLM application
21mBuilding RAG With OpenSearch ML Plugins: From PDFs To Voice-Enabled Search
33mTranslate tech talks using open-source models and cloud-native Wasm