The real energy cost of AI: Lessons from Google
Summary
Analysis of Google's data on Gemini AI energy use, highlighting the difference between chip-only and system-wide measurement and rapid efficiency improvements.
Key quotes
Google’s more comprehensive approach to calculating AI’s energy consumption shows that typical estimates can miss more than half of the energy actually used.
Over the past year, Google achieved a 33 times reduction in energy use per prompt through a combination of software and hardware improvements.
The article discusses the methodology of measuring AI energy consumption, comparing narrow chip-only metrics with comprehensive system-wide tracking. It notes Google’s efficiency gains via Mixture of Experts and custom TPU co-design.