An ASHRAE technical white paper on the factors accelerating the transition of data centers from predominantly air-based to liquid cooling. The document links rising CPU, GPU, FPGA, and memory power levels to the limitations of air-based systems in terms of airflow, temperature, noise, rack density, and personnel safety. It examines ASHRAE water classes, direct liquid cooling and immersion cooling, and the implications for PUE, TCO, power delivery, heat reuse, and deployment in enterprise, HPC, and colocation data centers.
Key Takeaways
Increasing socket and memory power, alongside declining allowable component case temperatures, limits the applicability of air cooling and increases the number of server SKUs that require liquid cooling.
Required airflow rates for dense servers may exceed the capacity of typical raised-floor air delivery. Higher fan speeds increase energy consumption, noise, and risks to HDDs.
Liquid cooling can increase compute density and reduce the number of racks and the associated physical network and electrical infrastructure required for a given IT load. Once deployed, power delivery may become the primary constraint on density.
Lower required water temperatures resulting from higher thermal loads reduce the available economizer hours and may increase CAPEX and OPEX. Loop temperatures should therefore be assessed in light of CPU/GPU and memory requirements, as well as the server's hydraulic architecture.
Large-scale deployment requires design provisions for piping and leak protection. In colocation facilities, SLA restrictions on water at the rack may make intermediate liquid-to-air solutions a practical first step.
Key Figures
Document publication year
2021
Stated in the ASHRAE copyright notice.
Period when moderate growth in server power consumption ended
about 2018 year
After this, rising power demand from compute, memory and storage subsystems began to complicate data center operations more significantly.
Lower limit of ASHRAE water classes
2 °C
The lower temperature limit for all W classes.
ASHRAE water classes
W17, W27, W32, W40, W45, W+
The designations reflect the upper temperature limit of the respective class.
Maximum temperature of ASHRAE air class A4
45 °C
113°F.
H1 class temperature range
15–25 °C
Class for air-cooled high-density IT equipment.
Socket power threshold that complicates air cooling of 1U/2U servers
300–400 W
At this power level, standard 1U and 2U servers become more difficult to cool with air.
Maximum airflow through a floor tile
1900 cfm
The stated level for best-in-class data centers.
Required airflow for individual servers
100 and more cfm/U
With 1900 cfm from one tile, a rack could be populated to only 19U.
Available rack height with one tile and airflow of 100 cfm/U
19 U
Calculated for 1900 cfm supplied through one floor tile.
Increase in power and cooling requirements of new servers
25 and more %
Compared with the servers being replaced during the “power wars” period.
Fan power share of server power
up to 20; in individual cases 2 %
Historically, the share declined through fan control and optimization; for dense servers, it may rise again.
Increase in air-fan speed
30–50 %
Causes a 6–9 dB increase in sound pressure level.
Increase in sound pressure level
6–9 dB
A consequence of a 30–50% increase in air-mover speed.
Acoustic risk range for HDDs
1–20 kHz
Acoustic disturbances in this range became the dominant risk mechanism for HDDs.
Sound pressure level that degrades HDD performance
over 110 dB
The unweighted overall sound pressure level can significantly degrade throughput.
Typical power of a high-density rack
40–50 kW
Such a rack may require up to 5000 cfm of airflow.
Power distribution voltage for large-scale HPC/liquid-cooled systems
480 V
A transition from 208 V can reduce conductor sizes, the number of circuits and PDUs, as well as distribution losses.
Office area that SuperMUC-NG waste heat can heat
40 000 m²
Equivalent to 430,500 ft².
Cooling capacity of the LRZ adsorption chiller
600 kW
Produces chilled water for rear-door heat exchangers.
Practical Value for Data Center Owners
For owners and design teams, the document provides guidance on when conventional air cooling is no longer sufficient: socket power, memory load, required airflow per rack unit, temperature constraints, and allowable rack density must be assessed together. During the concept and design stages, the architecture should reserve provision for liquid cooling, including distribution routes and rack connections, heat exchangers, water treatment, controls, leak detection and drainage, and residual air cooling. Economic comparisons should consider not only PUE, but also the density threshold at which liquid cooling reduces initial cost per MW, as well as the effect of water temperature on free-cooling hours and chiller requirements.
A special World Energy Outlook report by the International Energy Agency examining the relationship between AI development and energy. It analyses growth in data-centre electricity consumption and capital expenditure, the physical constraints on scaling AI infrastructure, power-supply options, and implications for energy systems, prices and emissions. It also considers AI’s impact on energy and industrial efficiency, as well as demand scenarios through 2030.
AI data centersData center power supplyGrid interconnectionLiquid cooling