In NVIDIA’s announcement at Computex 2022 tonight, the company revealed that it is preparing to launch a liquid-cooled version of its high-end PCIe accelerator card. The liquid cooling card, which is offered as an alternative to the traditional dual slot air cooling card, is offered in a more compact single slot form factor to improve both cooling and density. The liquid-cooled A100 will be available in the third quarter and the liquid-cooled H100 will be available early next year.
Liquid cooling is nothing new in the data center, but it is usually reserved for more bespoke hardware with extreme cooling and density requirements, such as next-generation high-end NVIDIA H100 (SMX) servers. In contrast, PCIe servers are all about standardization and compatibility. For server graphics cards / accelerators, it means dual slot cards designed for forced air cooling within the server chassis. This is useful for the market segment, but the 300-350 watt TDP of these cards means they can’t be thinned and are effectively cooled by air. This imposes a four-card limit on standard rack-mounting systems.
However, times are changing, and liquid cooling is being implemented in larger data centers to accommodate the cooling of increasingly hot hardware and improve the energy efficiency of the entire data center. To that end, NVIDIA has released liquid-cooled versions of the A100 and H100 PCIe cards, providing a simple and officially supported path for data center customers to install liquid-cooled PCIe accelerators in their facilities.
The card (pictured above) is basically a reference A100 / H100, replacing the traditional dual-slot heatsink with a single-slot full coverage water block. Designed to be integrated by server vendors, it uses an open-loop design intended for use as part of larger liquid cooling setups.
However, the card specifications have not changed, other than changing the cooling system. NVIDIA hasn’t increased the TDP or clock speed of these cards, so their performance should be the same as traditional air-cooled cards (unless, of course, they are thermally throttled). In other words, these new cards use liquid cooling to improve energy efficiency and density, not performance.
The first card out of the gate will be a liquid-cooled version of the 80GB A100 PCIe Accelerator. It will be available to customers in the third quarter of this year. Meanwhile, a liquid-cooled version of the H100 PCIe is under development, and NVIDIA expects it to be available in early 2023.
Tentatively, NVIDIA is working with Equinix to certify the liquid-cooled A100 in the data center for ideas on the actual power savings of new hardware. Interestingly, NVIDIA reports that switching to liquid cooling has significantly reduced power usage across the data center. With a 2000 server (4000 A100 card) setup, total power demand was reduced by 28%. According to NVIDIA, this is due to the combination of overall power savings across the data center from the switch. This includes everything from improving the energy efficiency of graphics cards due to low temperatures to reducing energy demand through the operation of cooling water and large air chillers. All of this highlights why NVIDIA is pushing liquid-cooled hardware to improve power efficiency for data center operators looking to reduce power usage.
And while this first-generation liquid-cooled hardware focuses on efficiency, it’s not always the case, according to NVIDIA. For future generations of cards, we will also consider liquid cooling to improve performance at current energy levels, perhaps by investing data center-scale profits in the card’s higher TDP.
Finally, while most of today’s announcements (and case studies) focus on PCIe cards, NVIDIA also reveals that it is also working on the official liquid cooling design for HGX systems. Accommodates the company’s more powerful SMX card. The liquid-cooled HGXA100 has already shipped and the liquid-cooled HGXH100 will be available in the fourth quarter.
