Creating Your Own LLM Tuning Platform with Open Source Technologies
summary
An open source pipeline called FMS HF Tuning wraps HuggingFace SFT Trainer and PyTorch/FSDE for distributed runs, exposes a CLI sft_trainer.py and a Jupyter flow on Open Data Hub/Red Hat OpenShift AI, and supports LoRA, prompt tuning and full fine‑tuning. Demo fine‑tuning a tiny‑llama 1B for Yelp sentiment converts CSV→JSON, trains with 700,000 examples (vs noisy 200‑example runs), runs small jobs on an M3 laptop and larger jobs on A100 nodes, and produces repeatable models and successful PyTorch job deploys.
AI-written summary · about
chapters (7)
- 0:00LLM tuning project intro
- 1:03LLM improvement methods
- 6:44FMSF tuning library
- 11:43Sentiment analysis demo
- 18:09OpenShift model deployment
- 19:10OpenShift deployment demo
- 26:52Q&A
related talks
16mStreamline LLM Fine-tuning on Kubernetes With Kubeflow LLM Trainer
29mLocal LLM fine-tuning: a practical example
21mStreamline LLM Fine-Tuning on Kubernetes with Kubeflow LLM Trainer
32mRunning and Fine-tuning Open Source LLMs on Google Kubernetes Engine
26mVLLM-Helion: SOTA LLM Performance by Advanced Autotuning and Fine-grained Dispatching
39mThe Future is Fine-Tuned: Deploying Task-specific LLMs
25mFine-Tuning Large Language Models With Declarative ML Orchestration
40mInstructLab: Applying Open Source Methods to Building and Trai...