Tame your Llama, run AI in Java
summary
Java runs local LLMs with practical, runnable code using Olama4j + Olama server and LangChain4j, creating custom models like a “llama animal expert” and a Viking expert (Yoda style) that respond live but sometimes hallucinate. Example: model creation completes in ~0.6s, prompt uses 45 tokens and completion 194 (total ~293). Minimum workable hardware is ~16GB RAM, 8 cores, 500GB; GPUs speed things up. Memory and chat history are the common failure points; offline privacy and cost control are the main gains.
AI-written summary · about
chapters (12)
- 0:00Welcome
- 2:03Speaker and conference
- 4:30Large language models
- 10:18Local vs hosted models
- 15:42Java AI with Ollama
- 18:38Live demo: local LLM in Java
- 23:57Custom AI experts in Java
- 29:24LangChain4j for AI workflows
- 33:22Ollama4j vs LangChain4j
- 40:28Hardware and model guide
- 43:47Local LLMs takeaways
- 45:57Resources and Q&A
related talks
51mPractical LLM Inference in Modern Java
44mPimp My Llama - How to Customize LLMs for your Business Case
47mOwn Your LLM: Running Generative and Agentic AI Locally
51mJlama: A Native Java LLM inference engine
46mThe Era of AAP: Ai Augmented Programming using Modern Java
55mBuild enterprise generative AI apps using Llama 3 at 1,000 tokens/s on the SambaNova AI platform
47mPractical LLM Inference in Modern Java
51mInternal AI solutions -from hype to production