LLM: Efficiency and Sustainability in Practice
summary
The talk compares environmental and efficiency metrics for large models (e.g., 20,000–50,000 tonnes CO2 to train big models, ~15 g CO2 and ~50 ml water per ChatGPT request, with Google Gemini much lower) and presents practical methods to reduce cost and footprint: quantization, pruning, and knowledge distillation. It shows quantized generative models can run on CPUs at far lower cost, pruning speeds up token generation, and a distilled pipeline (3,000 LLM‑labeled records used to train an SVM) achieves about 90% of GPT‑4 accuracy while running on mobile hardware, with working demos provided.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:12LLM adoption impact
- 5:30Energy and water costs
- 8:42Efficient usage strategies
- 9:23Model size reduction
- 11:53Quantized models use cases
- 13:16Model distillation
- 15:02Sustainable initiatives
- 16:50Prompt optimization
- 18:16Conclusion
related talks
27mProductionizing Open Source LLMs
24mHarnessing the Power of LLMs in Advanced Analytics
19mThe Quest for the Greenest LLM
25mAggressive LLMs Optimization: Making Them Work on Tiny Devices
28mAre LLMs the answer to all our problems?
39mCollect Data to Fine-Tune Your Large Language Model
19mSystem for Scaling Large Language Models
33mThe Right Model for the Right Job: Cutting LLM Costs Without Cutting Quality