RamaLama: Making working with AI Models Boring
summary
RamaLama packages open-source LLMs into reproducible containers using Podman or Docker, pulls models from Ollama, Hugging Face or OCI, and runs llama.cpp or vllm under the hood; demos include a 4B vision model and a 135M chat model, benchmarking about 1300 tokens/sec prefill and 60–70 output tokens/sec versus ChatGPT’s ~20–50. It automates RAG with Dockling → vector DB, produces quadlet and Kubernetes YAML for deployment, and enforces container isolation flags (no network, read-only FS, auto cleanup).
AI-written summary · about
chapters (16)
- 0:00RamaLama intro and local AI
- 3:31Live demo with camera
- 8:28AI tools comparison
- 11:57Llama C++ vs VLM runtimes
- 14:23RamaLama and containers
- 15:26Demo pulling models
- 18:16GPTOSS with vector data
- 20:01RamaLama security
- 21:44Model registries verification
- 23:51RAG and Dockling
- 26:42RAG demo with Amtrak PDF
- 29:10Deploying with systemd Quadlets
- 31:44Deploying to Kubernetes
- 33:31Agentic AI with Java
- 37:25Blackjack AI demo
- 42:26Q&A
related talks
35mRamaLama: Running AI Models in Containers.
37mWhat's new with RamaLama
12mLightning Talk: Ramalama - The Project That Brings AI and Containers Together While Making AI Boring
32mGet Up and Running with Llamastack to Create AI Applications!
17mUnpacking the Llama Stack: Architecting Next-Gen AI Applications
45mEverything You Need to Know About Running LLMs Locally
27mAI Demystified Unveiling the Intelligence Behind the Machine
01h 19mBuilding an LLM on your Laptop to...