Building Your own AI Assistant with JavaScript, LLMs, and Rag
summary
JavaScript code ties local or hosted LLMs to a vector database via embeddings so private docs get pulled into a retrieval augmented generation pipeline; LM Studio and containerized models give an OpenAI‑style API, and a Llama 3.2 2‑billion‑parameter model runs on a laptop with ~3GB safe‑tensor slices. Recommendation: aim for ~1–2B params for useful accuracy; tiny models (0.5B) hallucinate confidently (example: answers “Florence” for Italy’s capital). The prototype works but needs RAG to be reliable for private data.
AI-written summary · about
chapters (15)
- 0:00Building an AI assistant
- 1:49AI buzzword overwhelm
- 3:36AI engineers vs AI chefs
- 5:23AI model request processing
- 7:09AI model fundamentals
- 9:13Model parameters and resources
- 12:01Model knowledge cutoffs and private data
- 13:07Embeddings and vector spaces
- 14:29Retrieval augmented generation
- 15:33Hosted AI models like ChatGPT
- 16:37JavaScript demo small model
- 18:02Live demo inaccuracies
- 20:10LM Studio local models
- 21:57Model accuracy versus cost
- 23:09Q&A
related talks
27mBuilding Useful Apps With Small, Local LLMs
47mOwn Your LLM: Running Generative and Agentic AI Locally
27mAI Demystified Unveiling the Intelligence Behind the Machine
40m4 pillars for building a modern AI application in JavaScript
31mYou are an AI engineer
40mPractical Lessons from Deploying AI Assistants Based on LLMs
47mRamaLama: Making working with AI Models Boring
18mLLMs and GenAI: A Guide to Building Your Own RAG Model for Document Interaction