Beyond Autocomplete: Local AI Code Completion Demystified
summary
JetBrains ships a local AI code completion that runs a 100M-parameter Llama model (1500-token context) quantized to 4 bits occupying ~50–500MB and using lama CPP for inference; median time-to-suggestion is ~300–400ms and full model training costs about $5,000. A/B tests show fraction of code generated rises 30–100% while a smart filter model increases acceptance 20–50% and halves explicit cancels at the cost of ~20% fewer total selections. Suggestions pass IDE correctness checks before display.
AI-written summary · about
chapters (14)
- 0:00Local AI code completion intro
- 1:09JetBrains IDEs overview
- 4:20Feasibility of local AI
- 6:27High-level architecture
- 7:29Online A/B testing
- 10:16User experience metrics
- 11:44Post-processing pipeline
- 14:37Smart filtering for acceptance
- 17:06Model inference improvements
- 18:54Language model training
- 20:41Offline evaluation
- 22:07Team structure
- 22:50Key takeaways and future
- 23:33Q&A
related talks
34mHow to train a fast LLM for coding tasks
41mWill AI Assistant make developers redundant?
49mTame your Llama, run AI in Java
46mThe AI-Powered Developer
41mAI Tooling for Scala Developers - What JetBrains Brings to the Table
42mUsing LlamaIndex for building Generative AI apps
51mPractical LLM Inference in Modern Java
11mllama.cpp - faster and easier for everyone!