What's new with RamaLama
summary
Project demonstrates running models locally in containers with Podman, auto-detecting GPU type and pulling a matching image, dropping privileges for isolation, and exposing an OpenAI‑compatible REST API or Llama Stack endpoints. Commands like ramalama run and ramalama serve pull preselected short‑name models, default to Llama CPP (also supports whisper CPP), build RAG pipelines using Dockling and Quadrant, package models and data as an OCI artifact, and can emit Kubernetes YAML or quadlets for cloud/edge deployment; repo has 100+ contributors and ~2.2k stars.
AI-written summary · about
chapters (8)
- 0:00RamaLama project intro
- 4:59Project goals and values
- 7:07Containers and GPUs
- 13:32Llama CPP vs VLM
- 17:07Live demo features
- 27:06Models to OCI artifacts
- 31:57RAG databases and embeddings
- 35:06Q&A
related talks
35mRamaLama: Running AI Models in Containers.
47mRamaLama: Making working with AI Models Boring
12mLightning Talk: Ramalama - The Project That Brings AI and Containers Together While Making AI Boring
32mGet Up and Running with Llamastack to Create AI Applications!
17mUnpacking the Llama Stack: Architecting Next-Gen AI Applications
45mEverything You Need to Know About Running LLMs Locally
11mIntroducing Llama API: A Seamless Way to Revolutionize Your Workflow
15mStreamlining Investment Decisions with Python, LangChain, and RamaLama