The elephant in the server room
I’ve spent the last two years helping businesses integrate generative AI into their operations. And there’s one conversation that keeps getting pushed to the bottom of the agenda: energy.
It’s understandable. When you’re trying to work out how AI fits into your content pipeline or your customer service workflow, the carbon footprint of a data centre in Iowa doesn’t feel like your problem. But it is. And it’s going to become a bigger one.
Here’s the reality. Generative AI models are getting larger. GPT-4 was estimated to have been trained on around 25,000 GPUs running for months. The energy consumption figures for these training runs are staggering - and that’s before you factor in inference, which is the energy used every single time someone sends a prompt.
For businesses scaling their AI usage, this isn’t just an environmental concern. It’s a cost concern. And increasingly, it’s a reputational one.
The scale of the problem
Let me put some numbers on this. A single query to a large language model uses roughly 10 times the energy of a standard Google search. That might not sound like much until you multiply it across an organisation running thousands of AI-assisted tasks per day.
Training costs are even more dramatic. Training GPT-3 consumed an estimated 1,287 MWh of electricity. Newer, larger models dwarf that figure. And every time a model is fine-tuned, retrained, or updated, that energy bill resets.
The water usage is significant too. Data centres need cooling, and that means water consumption measured in millions of litres. Microsoft reported a 34% increase in water consumption in 2023, largely attributed to AI workloads.
None of this means we should stop using AI. That ship has sailed, and frankly, the productivity gains are too significant to ignore. But it does mean we need to be smarter about how we use it.
What mid-size businesses can actually do
I’m not going to tell you to lobby governments or invest in renewable energy infrastructure. You’re running a business, not a campaign. Here’s what’s actually within your control.
1. Right-size your models
This is the single biggest efficiency gain most businesses can make. Not every task needs GPT-4. A smaller, more efficient model - or even a fine-tuned open-source model - will handle 80% of your use cases at a fraction of the compute cost.
I’ve worked with agencies running everything through the most expensive API tier when a lighter model would produce identical results for their specific tasks. That’s wasted energy and wasted money.
The principle is simple: use the smallest model that does the job well.
2. Optimise your prompts
Badly structured prompts generate more back-and-forth, more token usage, and more compute. A well-designed prompt system with clear instructions, examples, and constraints gets better results in fewer cycles.
This isn’t just good practice - it’s directly linked to energy efficiency. Fewer tokens processed means less compute, less energy, and lower costs. I’ve seen prompt optimisation reduce API costs by 40-60% for some clients. The environmental benefit follows the same curve.
3. Cache and reuse intelligently
If you’re generating the same types of output repeatedly - product descriptions, social copy, email templates - you don’t need to hit the API every time. Build caching layers. Store and adapt previous outputs. Use AI for the creative heavy lifting, then templatise the results.
4. Choose your providers deliberately
Not all AI providers are equal on sustainability. Some run on renewable energy. Some don’t. Some publish their energy consumption data. Others treat it as commercially sensitive.
Ask the question. When you’re evaluating AI tools, add energy efficiency and data centre sustainability to your criteria. It matters, and the providers who take it seriously deserve the business.
5. Audit your usage
Most businesses have no idea how many API calls they’re making, what they’re costing, or what the compute footprint looks like. Build visibility into your AI usage. Track it monthly. Look for waste.
I guarantee you’ll find tasks where AI is being used unnecessarily - where a simple template, a database lookup, or a human decision would be faster, cheaper, and more appropriate.
The commercial case
Here’s what I find interesting: almost everything on that list also saves money. Right-sizing models reduces API costs. Prompt optimisation cuts token usage. Caching reduces redundant calls. Auditing eliminates waste.
Sustainability and commercial efficiency aren’t in tension here. They’re the same thing. The businesses that figure this out first will have leaner, cheaper AI operations and a genuine story to tell about responsible adoption.
The moving finish line
The technology is shifting fast. Model efficiency is improving with every generation. Techniques like quantisation, distillation, and mixture-of-experts architectures are making it possible to get more capability from less compute.
But adoption is scaling faster than efficiency is improving. More businesses, more use cases, more queries, more compute. The net energy demand is going up, not down.
That’s why this matters now. Not in five years when regulation catches up. Not when your clients start asking about it - though they will. Now, while you’re building your AI operations, is the time to build them efficiently.
My take
I’m not an environmentalist. I’m a pragmatist. I help businesses adopt AI because it works. But I’ve seen too many organisations treat compute as infinite and free. It’s neither.
The businesses that scale AI sustainably won’t just have a smaller carbon footprint. They’ll have lower costs, more efficient workflows, and a competitive advantage when sustainability reporting becomes mandatory for their sector.
Start with the model. Right-size it. Optimise around it. Audit regularly. It’s not complicated - it just needs to be intentional.
