MuxTune: Efficient Multi-Task LLM Fine-Tuning in Multi-Tenant Datacenters...
summary
Datacenter fine-tuning system that multiplexes a frozen LLM backbone across concurrent PEFT tasks, using spatial-temporal multiplexing and a hierarchical co-scheduling scheme (task/operator/data levels). Uses LoRA-style adapters decoupled via PyTorch hooks, intra-stage subgraph scheduling with separate CUDA streams, and a static pipeline template to cut bubbles. Chunk-based trunk alignment removes inter-task padding. Claims up to 2.33× throughput and 5.29× memory reduction; notes PEFT shows a 1.47× MFU gap and naive pipeline/tensor-parallel setups slow 1.16–1.17×.
AI-written summary · about
chapters (11)
- 0:00MuxTune introduction
- 1:22PEFT performance challenges
- 3:52Existing multiplexing limits
- 5:37Spatiotemporal multiplexing
- 7:25MuxTune system design
- 8:28Sharing frozen backbone
- 9:34Fusion trade-offs
- 10:37Operator orchestration
- 11:39Pipeline scheduling
- 12:42Chunk-based alignment
- 13:44Evaluation results
related talks
15mResource Multiplexing in Tuning and Serving Large Language Models
12mMemory-Efficient AI: How PEFT and PyTorch Enable Accessible LLM Fine-Tuning
39mThe Future is Fine-Tuned: Deploying Task-specific LLMs
35mCreating Your Own LLM Tuning Platform with Open Source Technologies
16mtorchtune: Easy and Accessible Finetuning in Native PyTorch
20mFwdLLM: Efficient Federated Finetuning of Large Language Models with Perturbed...
10mAuto-tuning vllm
26mEfficiently serving LLMs at scale.