Nvidia announced its new dual-GPU offering, the H100 NVL, during its GTC Spring 2023 keynote. This won’t revive his SLI or multi-GPU gaming, nor will it be one of the best graphics cards for gaming, but instead target the growing AI market. According to information and images released by Nvidia, the H100 NVL (H100 NVLink) has three of his NVLink connectors on top, with his two adjacent cards going into separate PCIe slots.
An interesting change of pace to accommodate servers that don’t support Nvidia’s SXM option, with a focus on inference performance rather than training. The NVLink connection helps provide the missing bandwidth that NVSwitch offers in his SXM solution. There are some other notable differences as well.
take the specs. The previous H100 solution (both SXM and PCIe) came with 80GB of HBM3 memory, but the actual package contains 6 stacks, each with 16GB of memory. It’s not clear if one stack is completely disabled or for ECC or other purposes. What we do know is that the H100 NVL will have HBM3 with 94GB per GPU and 188GB total. I’m assuming the “missing” 2GB per GPU is currently for ECC.
Power is slightly higher than the H100 PCIe at 350-400 Watts per GPU (configurable), a 50W increase. Meanwhile, the total performance is effectively doubled over the H100 SXM: 134 teraflops on FP64, 1,979 teraflops on TF32, 7,916 teraflops on FP8 (and 7,916 teraflops on INT8).
Basically, this looks like the same core design as the H100 PCIe that also supports NVLink, but with more GPU cores enabled and a potential 17.5% increase in memory. Memory bandwidth is also significantly higher than the H100 PCIe at 3.9 TB/s per GPU for a total of 7.8 TB/s (2 TB/s for the H100 PCIe and 3.35 TB/s for the H100 SXM).
Since this is a dual card solution, Nvidia only supports 2-4 pairs of H100 NVL cards for partner and certified systems as each card takes up 2 slot spaces. How much does a pair cost and can they be purchased separately? Single H100 PCIe can sometimes be found, but that has not been confirmed yet. about $28,000 (opens in new tab)So $80,000 for a pair of H100 NVL doesn’t seem out of the question either.
