AMD Expands AI/HPC Product Lineup With Flagship GPU-only Instinct Mi300X with 192GB Memory

As part of today’s AMD Datacenter event, in addition to EPYC server CPU updates, the company also provides an update on the status of the nearly completed AMD Instinct MI300 accelerator family. The company’s next-generation HPC-class processors use both Zen 4 CPU cores and CDNA 3 GPU cores in a single package and are now his XPUs in a multi-SKU family.

Along with the previously announced 128GB MI300 APU (now called MI300A), AMD is also producing pure GPU parts using the same design. Called the MI300X, the chip will only use CDNA 3 GPU tiles instead of the MI300A’s combination of CPU and GPU tiles, making it a pure high-performance GPU paired with 192 GB of HBM3 memory. The MI300X is directly targeted at the large language model market and is designed for customers who need all the available memory capacity to run their largest models.

First announced last June and further detailed at CES 2023, the AMD Instinct MI300 is AMD’s big take on the AI ​​and HPC markets. A unique server-grade his APU features both Zen 4 CPU cores and CDNA 3 GPU cores in a single chiplet-based chip. AMD’s competitors this year don’t have (or plan to have) a combined CPU and GPU like the MI300 series, so AMD is looking for a truly unified memory architecture and plenty of bandwidth between the CPU and GPU tiles. It offers a range of interesting solutions.

MI300 also includes on-chip memory via HBM3 with 8 stacks. At the time of the CES announcement, the largest HBM3 stack was 16 GB, resulting in a chip design with a local memory pool of up to 128 GB. However, thanks to the recent introduction of his 24GB HBM3 stack, AMD is now able to offer a 192GB version of his MI300 with 50% more memory. This, together with the additional GPU chiplet featured on the MI300X, is intended to be a powerful means to handle the largest and most complex LLMs.

Under the hood, the MI300X is actually a slightly simpler chip than the MI300A. AMD replaced the MI300A’s 3 CPU chiplets with just 2 CDNA 3 GPU chiplets, resulting in a total of 12 chiplet designs, believed to be 8 GPU chiplets and an additional 4 IO memory chiplets. Otherwise his MI300X with GPU only looks a lot like the MI300A, albeit removing the CPU cores (and de-APUing the APU). And clearly AMD is aiming to take advantage of synergies by offering both its APU and flagship CPU in the same package.

Raw GPU performance aside (we don’t have concrete numbers at the moment), part of the story for AMD’s MI300X will be memory capacity. Given that memory capacity is the limiting factor for the current generation of Large Language Models (LLMs) for AI, even offering a single 192 GB chip is a big deal. As we’ve seen with recent developments from NVIDIA and others, AI customers are getting GPUs and other accelerators as soon as possible, while demanding more memory to run even larger models. So being able to offer his massive 192GB GPU with 8 channels of his HBM3 memory would be a huge advantage for AMD in the current market, at least once his MI300X starts shipping.

The MI300 family is on track to ship at some point later this year. According to AMD, the 128GB MI300A APU is already sampling to customers. On the other hand, the 192GB MI300X GPU will be sampled to customers in Q3 of this year.

It goes without saying that this announcement also confirms that AMD has a flexible XPU design at least three years ahead of rival Intel. While Intel has discontinued its Falcon Shores product, which combines CPU and GPU, and developed a pure GPU Falcon Shores, AMD plans to offer a flexible CPU+GPU/GPU only product at the end of this year. During this period, it will compete with products such as NVIDIA’s Grace Hopper superchip. This superchip isn’t even his APU/XPU, but it gets very close by linking his NVIDIA’s Grace CPU to his Hopper GPU via his high-bandwidth NVLink. So while we wait for more details on the MI300X, it should be a very interesting battle between the two his GPU giants.

Overall, the pressure on AMD with the MI300 family is significant. Demand for AI accelerators has seen no ceilings for most of the past year, and the MI300 marks the first time his AMD has played a significant role in the market. The MI300 isn’t a winner or loser for the company, but in addition to getting the technical advantage of being the first to ship a single-chip server APU (and the bragging rights that come with it), you also get: A fresh product to sell to a market that is buying up all the hardware available. That said, the MI300 is expected to be AMD’s money-printing license (similar to NVIDIA’s H100), and AMD’s keen investors hope so.

AMD Infinity Architecture Platform

In tandem with today’s 192GB MI300X news, AMD also briefly announced what it calls the AMD Infinity Architecture Platform. This is an 8-way MI300X design that can link up to 8 AMD’s top-end GPUs together to handle bigger workloads.

As we’ve seen with NVIDIA’s 8-way HGX boards and Intel’s own x8 UBB for Ponte Vecchio, 8-way processor configurations are currently the sweet spot for high-end servers. This is for physical design reasons: space to place the chips and space to wire the cooling through the chips, and to link a large number of chips without too many hops between chips. This is due to both of the best topologies available for This is another area where AMD will need to match his NVIDIA’s hardware offerings if AMD is going to go head-to-head with his NVIDIA and grab a piece of the HPC GPU market.

AMD calls the Infinity Architecture Platform an “industry standard” design. According to AMD, it uses the OCP server platform as a base here. This implies that the MI300X uses his OAM form factor, but we are still waiting to get explicit confirmation of this.