Average rack power density has more than doubled in two years, from 8 kW to 17 kW, and is projected to reach 30 kW by 2027, according to an October 2024 McKinsey report, with AI training racks already well ahead of that average.
Those limits show up in GPU clock speed. H100 GPUs under inadequate air cooling can throttle to a fraction of their rated clock speed within seconds of a sustained training run. In distributed jobs across thousands of GPUs, one throttled chip can stall the entire run. The DOE estimates cooling accounts for up to 40% of data center energy use.
JLL research establishes three density thresholds:
- Up to ~20 kW per rack: air cooling is adequate
- Up to ~100 kW: rear-door heat exchangers extend viability
- Above ~175 kW: immersion cooling is required
Direct-to-chip cooling fills the middle band, handling densities between ~100 and ~175 kW where rear-door exchangers fall short and immersion is not yet warranted.
Hot water changes the economics
Mechanical chillers are one of the biggest energy draws in any liquid-cooled data center, and until recently there were an unavoidable cost of liquid cooling. Nvidia’s Vera Rubin processor is changing that.
At CES in January 2026, Jensen Huang announced that Vera Rubin supports liquid cooling at 45 degrees Celsius, high enough for data centers to reject heat through dry coolers using ambient air rather than mechanical chillers. Nvidia’s CES press release confirmed Rubin is in full production, with customer availability in the second half of 2026. According to Nvida’s product specifications, the Vera Rubin NVL72 uses warm-water, single-phase direct liquid cooling at a 45°C supply temperature, allowing data centers to reject heat through dry coolers using ambient air rather than energy-intensive chiller systems.
