AssyLLM: Efficient Federated Fine-tuning of LLMs via Assembling Pre-trained Blocks
summary
AssyLLM assembles pre-trained transformer blocks from multiple LLMs (LLaMA, OPT, etc.) so clients only run forward passes and upload block indices instead of gradients, cutting memory and communication. The system implements Block Comparator and layerwise CK/CR scores to pick compatible blocks, an Elastic Adapter to bridge dimension and semantics mismatches, a Block Quanter for per-block precision, and a Block Swapper for memory-aware swapping. On 28-block pools the pipeline assembles 21 models and reports up to 18.26% accuracy gain, 30x speedup, and 92% memory reduction.
AI-written summary · about
chapters (15)
- 0:00AssyLLM framework intro
- 0:41Federated fine-tuning challenges
- 1:47Memory constraints in fine-tuning
- 3:39Limitations of memory-saving methods
- 5:29AssyLLM assembling core idea
- 6:12AssyLLM workflow steps
- 8:49Challenges in assembling blocks
- 9:31Measuring block compatibility
- 12:20Elastic adapters for blocks
- 13:44Quantizing block pool memory
- 15:31Dynamic block swapping
- 16:53AssyLLM key components
- 17:36Experimental setup
- 18:41Performance improvements
- 20:06Future directions
related talks
20mFlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guarantees
01h 00mFedOpt for LLMs
25mvLLM: Easy, Fast, and Cheap LLM Serving for Everyone
29mLocal LLM fine-tuning: a practical example
20mFwdLLM: Efficient Federated Finetuning of Large Language Models with Perturbed...
11mWhen Less Is More: A Technical Overview of LLMs and the Strength of Smaller Models
16mWLB-LLM: Workload-Balanced 4D Parallelism for Large Language Model Training
35mThe State of vLLM