A submarine deep underwater has a peculiar problem.
It is a sealed metal vessel packed with people, machines and electrical equipment.
All of them produce heat, and unlike a normal building, a submarine cannot simply open a window and let cool air inside.
The deeper a submarine goes, the greater the water pressure around it becomes.
Roughly every 10 metres deeper underwater adds enough pressure to equal the pressure of all the air pushing down on us at sea level.
At around 300 metres underwater, the pressure outside can be roughly 30 times what we experience at sea level.
That is an enormous amount of force pushing against the hull from every direction. So the submarine has to remain tightly sealed.
But that creates another problem. The same hull that keeps the water out also keeps the heat in.
Fortunately, a submarine has one big advantage. It is surrounded by an almost unlimited amount of cold seawater.
Heat naturally moves from a warmer place to a cooler one. So the heat produced inside the submarine can be transferred to the much colder water outside.
In a way, the ocean becomes one giant cooling system.
In 2013, Sean James, an engineer working on data-centre technology at Microsoft, wondered if the same idea could work for computers.
James had served aboard a US Navy submarine, so the idea seemed logical to him.
A data centre has a similar problem. GPUs generate a lot of heat while they are working. That heat has to be continuously removed.
James and another Microsoft engineer, Todd Rawlings, circulated an internal white paper explaining the idea.
Microsoft decided to test this idea. That eventually led to Project Natick.
They placed 864 servers across 12 racks inside a sealed cylinder and lowered it into the sea near Scotland’s Orkney Islands.
It sat around 117 feet underwater for roughly two years.
The seawater around it helped remove the heat generated by the servers.
But Microsoft eventually stopped pursuing Project Natick at scale for practical and operational reasons.
Other companies have tried different versions of the same thing.
A Chinese company called Highlander, which builds data-centre and marine technology, developed an underwater data centre off the coast of Sanya in Hainan, southern China.
Meta did this with location. It built a large data centre in Lulea in northern Sweden, close to the Arctic Circle, where cold outside air can help cool servers for much of the year.
Google took a different route at its data centre in Hamina, Finland, which we discussed in Part 2.
Even in India, Reliance is trying to use desalinated seawater as part of the cooling strategy at its Jamnagar data-centre project.
These companies are using very different methods, but they are all trying to solve the same problem: heat.
As data centres become larger and more powerful, keeping the servers cool is becoming one of the biggest challenges of running them.
That brings us to the next biggest cost: cooling.
Why Cooling Is Getting More Expensive
In Part 2, we looked at how much electricity an AI data centre uses. Almost all of that electricity eventually turns into heat.
So if a server rack is using 120 kW of power, the cooling system has to remove roughly 120 kW of heat as well, continuously.
NVIDIA’s GB200 NVL72 shows how extreme this has become. It is one liquid-cooled rack containing 72 Blackwell GPUs and 36 Grace CPUs.
And the whole thing is surprisingly compact.
The rack is roughly 2.2 metres tall, 0.6 metres wide and just over 1 metre deep. Think of something roughly as wide as a household refrigerator, but taller and deeper.
Now look at how much heat is coming out of that space.
NVIDIA says the rack uses about 120 kW of power. Some tests show that it can use closer to 130-132 kW when running at full power.
To put that into perspective, a normal room heater uses around 1.5 kW.
So a single rack, occupying less than one square metre of floor space, can produce about as much heat as 80 room heaters running at full power at the same time.
That is the real cooling problem: not just how much heat is being produced, but how much of it is packed into such a small space.
Spread across the rack’s footprint, that works out to close to 190 kW per square metre.
For comparison, strong midday sunlight delivers roughly 1 kW per square metre.
So the heat being generated there is almost 200 times more concentrated than sunlight hitting the ground.
And a data centre can have dozens of these racks sitting next to each other.
The numbers become even more extreme when we look at the chips themselves.
A single NVIDIA H100 GPU can use around 700 watts. That is roughly the power used by a small electric heater, except all of that heat is being produced by a chip small enough to hold in your hand.
Modern AI chips can push around 50-100 watts of heat through every square centimetre of silicon.
At that level, the amount of heat concentrated on the chip starts to approach what engineers deal with in parts of rocket engines.
The difference is that an AI chip is expected to keep doing this continuously for years.
And the problem is getting worse with every new generation.
NVIDIA’s A100 GPU used around 400 watts in 2020. The H100 increased that to around 700 watts, and the B200 reaches roughly 1,000 watts.
The same trend appears at the rack level. An H100-based rack uses about 40 kW, while the GB200 NVL72 uses around 120-130 kW. The newer GB300 Blackwell Ultra rack reaches roughly 132-142 kW.
So in only a few generations, the heat produced by a single rack has more than tripled.
Older data centres were never designed for this.
Traditionally, they relied mostly on air cooling: cold air passed through the servers, absorbed the heat, and was then cooled and recirculated.
The problem now is that air is not very good at carrying heat.
Water can absorb far more heat than the same volume of air. In practical terms, one gallon of water can carry away roughly as much heat as thousands of cubic feet of air.
That means once racks become powerful enough, you simply cannot push enough cold air through them to keep the equipment cool.
Air cooling starts becoming difficult at around 50 kW per rack.
A GB200 rack can use around 120 kW, already more than twice that level.
And future racks are expected to go much higher.
NVIDIA’s Vera Rubin systems are expected to reach around 190-230 kW per rack, while later systems could eventually approach 600 kW.
At those levels, cooling the servers with air alone becomes extremely difficult.
And that is why AI data centres are increasingly moving towards liquid cooling.
Roshan Chutkey, Senior Fund Manager at ICICI Prudential AMC, has more than 18 years of experience across equity research, fund management and macroeconomic research.
We asked him to look at India’s data-centre buildout from an investor’s perspective: as more capacity gets built, where could the opportunity emerge?
Cooling is one place he is watching closely.
“As data-centre power consumption increases, cooling requirements will also rise. The cooling opportunity is approximately $2.1 billion for every GW of data-centre capacity.”
“Advanced cooling technologies such as direct-to-chip and immersion cooling are largely handled by global companies, while domestic companies are mainly into MEP and HVAC systems.”
Roshan Chutkey
Senior Fund Manager, ICICI Prudential AMC
Basically, more data centres mean more demand for cooling. That could create a sizeable opportunity for companies providing cooling, MEP and HVAC systems.
How cooling became a much bigger part of the bill
Cooling a data centre has always cost money. But AI is changing just how big that cost can become.
To see the difference, start with the overall construction bill.
In 2020, building a data centre cost about $7.7 million for every MW of capacity on average globally. By 2026, that had risen to around $11.3 million per MW.
That is a 47% increase in just six years. And cooling is one of the reasons.
To see how important these systems already are, look at where the non-IT construction money goes.
In a traditional air-cooled data centre, the mechanical systems, things such as chillers, fans, pumps and other equipment used to remove heat, account for roughly 22% of the construction cost.
But once a facility is designed for liquid cooling, that share can rise to around 33%.
Think about it this way.
If the construction cost was Rs 100, around Rs 22 would go towards mechanical systems in an air-cooled data centre.
With liquid cooling, that could become around Rs 33.
So cooling-related systems can go from taking up roughly one-fifth of the construction bill to nearly one-third.
“The power and renewable-energy ecosystem could represent an opportunity of around $3.5 billion for every GW of data-centre capacity.”
“For 1 GW of IT load, the power requirement including the supporting ecosystem is around 1.4 GW of round-the-clock data-centre power capacity. This implies 2.5 GW each of solar and wind capacity, and around 6 GWh of battery capacity.“
Roshan Chutkey
Senior Fund Manager, ICICI Prudential AMC
That gives a sense of how much has to be built around the servers.
Every new rack needs electricity. That electricity has to be generated, transmitted, backed up and delivered reliably to the servers. And as the computing capacity grows, much of that supporting infrastructure has to grow with it.
So when billions of dollars are spent on a new data centre, the money can ripple through a much wider set of industries than the companies making the chips.
Why?
Well, liquid cooling simply needs more equipment.
The liquid needs to reach the servers, take the heat away from them and then carry that heat somewhere else.
That means more pipes, pumps and cooling equipment have to be built around the servers.
And remember, we are still only talking about the building.
The GPUs, servers and other computing equipment are not included in these numbers.
An AI-optimised data centre in the US can cost around 7-10% more to build than a comparable air-cooled facility.
Some industry estimates put the additional infrastructure needed for liquid cooling at around $2.7-3.7 million per MW, depending on the cooling system and equipment being used.
That may not sound enormous when looking at just one MW. But AI data centres are increasingly being built at scales of tens or even hundreds of megawatts, so the extra cost quickly becomes significant.
Take a 100 MW AI data centre as an example.
At an average construction cost of $11.3 million per MW, building the facility would cost roughly $1.13 billion, before installing a single GPU.
Now add the liquid-cooling infrastructure.
At an additional $2.7-3.7 million per MW, a 100 MW data centre could require another $270-370 million for cooling infrastructure.
That is roughly Rs 2,400-3,300 crore.
Just for cooling.
Cooling used to be one of the systems needed to run a data centre.
With AI, it is becoming one of the biggest things you have to spend money on while building one.
Where all that money goes
The extra cost becomes easier to understand if you follow how the heat moves through a data centre.
First, the heat has to be taken away from the chips.
For this, data centres use cold plates placed directly on top of processors. Coolant flows through them and carries the heat away.
The coolant then moves through manifolds and into coolant distribution units, or CDUs, which circulate it between the server racks and the data centre’s larger cooling system.
All of this also needs a network of pipes, valves and leak-detection systems running through the facility.
And even after the coolant has carried heat away from the chips, the job is not finished.
The heat is still inside the data centre. It has simply been moved somewhere else.
So the facility also needs equipment such as chillers, cooling towers, heat exchangers and dry coolers to finally release that heat outside.
Once you put all these pieces together, the cost difference becomes much clearer.
Industry estimates suggest that the complete cooling system for a 1 MW air-cooled data centre can cost around $1.5-2 million.
For the same 1 MW facility using liquid cooling, the cost can rise to around $3-4 million.
So the cooling system itself can cost roughly twice as much.
And once you move beyond cooling, the infrastructure bill gets even broader.
“Electrical equipment could represent an opportunity of around $1 billion for every GW of data-centre capacity. This includes substations, medium- and low-voltage switchgear, power-distribution units and related electrical work.”
“Cables are another part of it. Around 1 MW of data-centre capacity could require roughly Rs 35 million of cables and wires, split between traditional cables and optical fibre.”
“And once all this equipment arrives, companies are still needed to design, install and integrate it. The EPC and industrial-engineering opportunity could be around $1.8 billion per GW.”
Roshan Chutkey
Senior Fund Manager, ICICI Prudential AMC
That is when the data-centre story starts looking less like a pure technology story and more like an infrastructure buildout.
The GPU may sit at the centre of the facility, but around it are transformers, switchgear, cables, cooling systems, backup power and engineering work.
And someone has to manufacture, install and connect all of it.
That is why the data-centre opportunity can extend well beyond semiconductor and cloud companies.
But there is another cost to consider.
All the numbers above assume the data centre is being built for liquid cooling from the beginning. Converting an older facility can be much harder — and more expensive.
Even a new data centre designed to support direct liquid cooling can cost around 5-10% more to build, according to Uptime Institute, because it needs additional cooling equipment, pipes and other infrastructure.
For an older data centre, the problem becomes bigger.
Many older facilities were designed for air cooling and were never built to carry liquid all the way to powerful AI server racks.
So converting them can require major changes to the building and its existing cooling system.
Two cooling bills are growing at the same time
All this spending is turning data-centre cooling into a large industry of its own.
McKinsey expects the global data-centre cooling market to reach around $40-45 billion by 2030.
Within this, liquid-cooling equipment alone could become a $15-20 billion market.
For comparison, the liquid-cooling market was worth only a couple of billion dollars in the mid-2020s.
So in around five years, the market could become roughly five to six times bigger.
Why water becomes important
There is another problem with cooling: water.
One of the easiest ways to remove a large amount of heat is through evaporative cooling. A cooling tower allows some of the water to evaporate, and when it does, it carries heat away with it. The remaining water becomes cooler and can be used again.
This can use less electricity than relying completely on mechanical chillers. But there is an obvious problem: the water that evaporates is gone, so fresh water has to keep coming in to replace it.
At the scale of a large data centre, this can become a lot of water.
Google’s data centres, for example, consumed around 8.1 billion gallons of water across its data centres and offices in 2024.
A single large data centre using evaporative cooling can also consume millions of gallons of water on a hot day.
Now, compared with GPUs or electricity, water itself is not particularly expensive.
That is not really the problem.
The problem is whether enough water is available in the first place.
A data centre may be drawing water from the same supply used by homes, farms and factories nearby.
If the area already has a water shortage, adding a facility that needs millions of gallons can become a very big local issue.
This is why data-centre companies also track something called Water Usage Effectiveness, or WUE.
It simply tells us how much water a data centre uses for the electricity consumed by its IT equipment.
The industry average in the reference estimate is around 1.8 litres of water for every kWh of electricity used by the IT equipment.
Amazon reported around 0.15 litres per kWh in 2024
Some closed-loop and air-cooled systems can bring direct water consumption much closer to zero.
So the cooling system a company chooses can make a massive difference.
Google’s site-level disclosures show how widely water consumption can vary between facilities. Its Council Bluffs, Iowa, data centre consumed around 1 billion gallons of water in 2024, while its air-cooled facility in Storey County, Nevada, consumed around 1.5 million gallons.
That is a difference of nearly 700 times.
That difference explains why companies are now experimenting with systems that use less water.
Microsoft, for example, is moving towards closed-loop cooling systems where the same liquid keeps circulating instead of being lost through evaporation.
But there is a catch here too.
If you use less water, you may have to use more electricity. Mechanical chillers can reduce water consumption, but they usually need more power.
Using seawater is another option. But if that seawater needs to be desalinated first, that process itself uses electricity.
So there is no cooling system that solves everything. Use less water, and you may use more electricity. Use less electricity, and you may need more water.
And if you want to cool very powerful AI racks, you may need more expensive equipment altogether.
This is why cooling has become much more than just keeping servers cold.
The choice of cooling system can decide how powerful the servers can be, how much electricity the data centre needs, how much water it consumes and even where the data centre can be built.
What all this spending could mean for India
Now step back and look at the scale of what could be built in India.
“India currently has around 1.5-1.6 GW of data-centre capacity. Another roughly 3.5 GW is already under construction, while land has been acquired for projects representing around 10 GW of capacity.”
“For every GW of data-centre capacity that gets built, the total addressable opportunity for Indian companies could be around $6-7 billion.”
“That opportunity extends across power generation, renewable energy, electrical equipment, cooling and HVAC, backup power, EPC and industrial engineering.”
Roshan Chutkey
Senior Fund Manager, ICICI Prudential AMC
Those numbers give us a sense of how large the build-out could become.
India has around 1.5-1.6 GW of capacity today, while another 3.5 GW is already under construction. Beyond that, land has been acquired for projects representing around 10 GW more.
But planned capacity is not the same as capacity that will definitely get built.
And that is where Chutkey sees an important risk.
“One risk to watch is the financial viability of large AI companies, because the pace of future data-centre investment ultimately depends on whether this level of spending remains sustainable.”
The opportunity therefore depends on an important assumption: that companies can continue spending at this scale.
If the economics of AI do not support that investment over time, some of the capacity being planned today may never get built.
And that brings us back to the question we started this series with.
What does it actually cost to build an AI data centre?
The answer is not just the price of the chips.
It is the cost of the entire physical system required to make those chips useful.
That is perhaps the more useful way to look at the AI data-centre boom.
Not simply as a story about who makes the GPUs or owns the data centres.
But as an infrastructure build-out involving an entire ecosystem of companies around them with an opportunity that could be very large if the spending continues.
Nice article
Very well explained