AI deployment strategies: balancing efficiency and environmental impact
summary
Claims that LLM apps currently emit about 0.03 megatons a day (~11 megatons/year) and that adding 2 million H100s sold this year and Grace Hoppers drives another ~12 megatons added this year lead to a projection of ~0.5 gigatons in five years unless architectures change. Practical takeaways: ask LLMs for emissions and they often refuse, RAG hides failure modes, training can overfit into a costly “database,” and cheaper options like CPU inference and PGVector as a real datastore are viable.
AI-written summary · about
chapters (13)
- 0:00Intro
- 1:30Climate pledge
- 2:20Burning coal
- 3:10CO2 vs bombs
- 5:00How much does it cost
- 5:30Will it compensate
- 6:00How big is it
- 6:30Technical talk
- 10:00Training ML
- 10:30Efficiency
- 11:50Solutions
- 12:20What is happening here
- 14:50Outro
related talks
28mNavigating LLM deployment: Tips, tricks, and techniques
01h 49mAI engineering with open access LLMs that lie, curse, and steal
47mLean Mean Green Machines - Architecting for a Greener Future
15mvLLM: Easy, Fast, and Cheap LLM Serving for Everyone
20mSelf-hosting AI LLMs - a beginners guide
12mVertical Mobility: Inference from MVP to Trillion Parameters
36mProduction-Ready AI Agents: From LLMs to Small Language Models
44mML in Production