Fine-tune LLMs from Kaggle Models using (Q)LoRA
summary
Shows hands-on fine-tuning inside Kaggle Models notebooks using quantized Low-Rank Adaptation: load a model, apply 4-bit quantization, insert LoRA layers and train with Transformers + TRL SFT. Example 1: Gemma-class ~2.6B params with LoRA rank 5 drops trainable weights to ~3.6M and finishes ~20 minutes (20 epochs) on a free P100. Example 2: Llama 3 quantized + LoRA raises sentiment accuracy from 0.37 to 0.87 after five epochs (≈1 hour). Code and notebooks are on Kaggle Hub.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:09What is Kaggle and its features
- 3:43About the speaker and resources
- 5:12Kaggle Models
- 8:08Fine-tuning purpose
- 10:38Parameter-efficient LoRA
- 13:55Quantization with LoRA
- 14:18Fine-tuning Gemma 2B
- 18:56Fine-tuning Llama 3
- 24:21Key takeaways and Kaggle’s generative AI capabilities
- 25:47Q&A
related talks
22mTraining a LLaMA in your Backyard: Fine-tuning Very Large...
42mPost-Training Small Language Models: the adventures of a practitioner
39mThe Future is Fine-Tuned: Deploying Task-specific LLMs
29mLocal LLM fine-tuning: a practical example
29mRunning Large Language Models on Laptops...
25mFinetuning Large Language Models (LLMs) for Security Log Detections
25mFine-Tuning Large Language Models With Declarative ML Orchestration
40mUnlocking Local LLMs with Quantization