Federated Learning in the Generative AI Era
summary
Proposes multi-headed LoRA (RAVAN) that freezes random A,B and trains small H heads so communication sends only H matrices; keeps 2.4M trainable parameters on a ViT with 50 clients and closes most of the gap to full fine-tuning on non‑IID data by increasing effective rank and handling computational heterogeneity. Introduces FedFisher that sends WI plus Fisher/K‑FAC approximations to account for curvature and yields ~15–20% gains on pretrained ResNet/ImageNet over naive averaging. Shows flexible block‑level one-shot model merging to trade storage for accuracy and a federated router* that learns sparse, decentralized model evaluations to pick cheaper accurate models per query.
AI-written summary · about
chapters (16)
- 0:00Federated learning intro
- 1:03Generative AI eras and data scarcity
- 3:08Federated learning for edge data
- 4:44Federated learning in training
- 5:06Fine-tuning challenges with LoRA
- 8:03Inexact aggregation in LoRA
- 18:19RAVAN method for LoRA
- 25:29RAVAN results on datasets
- 33:14Computational heterogeneity in RAVAN
- 35:06Pre-inference optimization with merging
- 38:17FedFisher for model merging
- 44:29Flexible merging for multi-task
- 50:50Efficient inference with routing
- 56:35Router training results
- 58:08Why federated learning matters
- 1:00:48Q&A
related talks
34mTraining without sharing: Is Federated Learning the answer to data privacy in AI?
42mFederated Learning: Unlocking the Values of Discrete Data for Enterprise
41mFederated Learning: Machine Learning on Decentralized Data
31mFrom Theory to Practice: Advances in Real-World Federated Learning
42mLearning Together, Distributed: An Introduction to Federated Learning
31mFederated Learning: So Close and Yet So Out of Reach
34mDecentralized Machine Learning
20mFwdLLM: Efficient Federated Finetuning of Large Language Models with Perturbed...