Tailoring Llama for Your Needs: Fine-Tuning & Evaluation
summary
Two new Llama API products are introduced: a fine-tuning tool that customizes Llama 8B to specific data and an evaluations tool that measures quality. Examples: Articul8’s fine-tuned 8B beats GPT‑4 on Verilog at about 1/130th of the inference cost, AT&T runs transcript summarization 10× faster and 90% cheaper, and Spotify gets a 14% task lift. Workflow: upload JSONL chat completions, reserve 20% for evaluation, pick graders (semantic, string, factuality with e.g. 70% pass), fine-tune in ~15 minutes and download or host the model.
AI-written summary · about
chapters (6)
- 0:00Introduction to Llama API
- 1:05Cost and speed benefits
- 2:34Accuracy and use cases
- 3:43Importance of evaluations
- 4:44Fine-tuning in practice
- 7:20Running evaluation jobs
related talks
53mLLM Quality Optimization Bootcamp
48mPracticalities of Fine-Tuning Llama 2 with AI Studio
39mThe Future is Fine-Tuned: Deploying Task-specific LLMs
29mLocal LLM fine-tuning: a practical example
36mBuilding & Deploying LLM Apps
44mPimp My Llama - How to Customize LLMs for your Business Case
50mPimp My Llama - How to Customize LLMs for Your Business Case
31mCustomizing LLMs: A Guide to local Fine-Tuning...