TACC’s Stampede3 Supercomputer Uses Intel’s Xeon Max with HBM2E and Ponte Vecchio

The Texas Advanced Computing Center (TACC) has unveiled its newest computing center. stampede Stampede3, a supercomputer for open science research projects. TACC expects Stampede3 to go live this fall and reach full performance in early 2024. This supercomputer will become a key component of the US National Science Foundation’s (NSF) ACCESS scientific supercomputing ecosystem, and from 2024 he is projected to serve the open science community through 2029.

The 3rd generation Stampede cluster built by Dell incorporates 560 nodes powered by Intel’s Sapphire Rapids generation Xeon CPU Max processors, with each node providing 56 CPU cores and 64GB of on-package HBM2E memory. Surprisingly, TACC plans to run these nodes in his HBM-only mode, so no additional DRAM will be connected to the CPU nodes and all memory will be sourced from the on-chip HBM stack.

With these specs, Stampede3 is expected to deliver a peak performance of around 4 FP64 PetaFLOPS and deliver around 63,000 general purpose cores. In addition, TACC will install 10 Dell PowerEdge XE9640 servers with 40 Intel Data Center GPU Max computing GPUs for artificial intelligence and machine learning workloads.

Given this layout, most of Stampede3’s computational performance is provided by the CPU. This makes Stampede3 a bit of a rarity these days. Most high-performance systems are GPU-driven, making Stampede3 one of the last supercomputers to rely almost exclusively on general-purpose CPUs.

While current clusters are primarily focused on CPU performance, TACC will also use Intel GPUs in the latest Stampede refinement and will investigate how future versions of the system will incorporate large amounts of GPUs. Currently, most of TACC’s AI tasks run on Lone Star systems with hundreds of Nvidia A100 computing GPUs. Therefore, the purpose of this organization is to investigate whether some of this workload can be transferred to Intel’s Ponte Vecchio.

“We plan to deploy a small system with exploration capabilities using Intel Ponte Vecchio. The exact amount is still being negotiated, but we believe it will be a minimum of 40 nodes and a maximum of 100 nodes,” said Dan Stanzione, executive director of TACC. […] We’re in the process of installing some Ponte Vecchio racks to see how people treat them. ”

Stampede3 leverages 400 Gb/s omnipath fabric technology to deliver 24 TB/s of backplane bandwidth. This setup allows you to efficiently scale your machine with minimal latency, making it ideal for a variety of applications that require simulation.

TACC also plans to re-integrate nodes from previous versions, Stampede2, based on older generation Xeon Scalable CPUs. This integration enhances his Stampede3’s capabilities for high-memory applications, high-throughput computing, interactive workloads, and other previous-generation applications. In total, the new supercomputer system will feature 1,858 compute nodes with over 140,000 cores, over 330 TB of RAM, 13 PB of new storage capacity, and peak performance approaching 10 petaFLOPS.

source: TACC, HPC wire