image 1 of 2
I visited AMD’s offices here in Taipei, Taiwan during Computex 2023 to speak with David McAfee, the company’s Corporate VP and General Manager of Client Channel Business. Our conversation covered a lot of topics that we write about separately, but we also got the chance to see AMD’s Ryzen XDNA AI engine in action in a laptop demo. McAfee also explained the steps AMD is taking to prepare its ecosystem of operating systems and software for the burgeoning AI use cases running locally on PCs. More on this below.
After following a hallway map inspired by AMD’s codenames, I went to the demo room to see AMD’s latest technology in action.
AMD’s demo laptop was the Asus Strix Scar 17, powered by AMD’s 4nm ‘Phoenix’ Ryzen 9 7940HS processor paired with Radeon 780M graphics. These 35-45W chips feature Zen 4 architecture and RDNA 3 graphics. AMD was running the same demo on the Asus ROG Zephyrus G14.
Our focus is on the XDNA AI engine, a dedicated accelerator that resides on-die with the CPU core. The overall goal of the XDNA AI engine is to run low-intensity inference workloads such as audio and video processing at lower power than can be achieved with CPUs and GPUs, while achieving faster response times than online services, It’s about improving performance and saving battery. Power.
The engine can handle up to four simultaneous AI streams, but can be quickly reconfigured to handle different amounts of streams. AMD claims that the engine, a descendant of Xilinx IP, is faster than the neural engine found in Apple’s M2 processor. It can handle up to 4 simultaneous AI streams, but can be quickly reconfigured to handle different amounts of streams. This engine is directly connected to the chip’s memory subsystem, so it shares a pool of coherent memory with the CPU and integrated GPU. The album above contains his AMD slides at the time of the announcement. These slides give an overview of how the engine works.
First, I opened Task Manager to see if the AI engine enumerates itself as a visible core with utilization metrics, but the XDNA AI engine does not show up as a visible device. In his manager he found the AI engine listed as an “AMD IPU device”, as seen in the album above. However, we were not able to observe any load or other telemetry from the core during testing.
image 1 of 1
Here you can see the XDNA AI engine running fast on the face recognition workload. The right side of the screen shows the measured latency for each step of the workload. The bar was impressively low, and the workload ran through a series of images quickly as the AI engine processed the inference workload. There is absolutely no explanation as to how these numbers compare to other types of solutions.
AMD’s demo had a button to test the onboard AI engine against their online Azure ONNX EP service, but the demo team said it had a software issue and didn’t work. Naturally, we would expect his built-in Ryzen AI engine to have lower latency than the Azure service. Logically, that’s what AMD was trying to demonstrate here. Unfortunately, I didn’t have an empirical point of comparison for the benchmark results.
However, benchmarks show that AI is alive and breathing on AMD’s Ryzen 7040 processors, and the company is also on track to increase the number of applications that can take advantage of its AI engine.
AMD announced last week at Microsoft’s Build conference that A new set of developer tools It leverages the open source Vitis AI Execution Provider (EP) and is upstreamed in the ONNX runtime, facilitating the work required to add software support for the XDNA AI engine. McAfee explained that Vitis AI EP acts as a kind of bare metal transformation layer, allowing developers to run models without modifying the base model. This simplifies integration.
AMD hasn’t provided performance metrics for its AI engine yet, but McAfee noted that it’s hard to quantify the benefits of an onboard AI engine with just one performance metric like TOPS. Other benefits are higher power efficiency and lower latency, which are some of the multifaceted benefits of having an AI engine. However, AMD plans to share numbers in the future.
McAfee continues to execute on its XDNA AI roadmap and reiterated the company’s plans to add the engine to other Ryzen processors in the future. However, the software ecosystem for AI on PCs is still in its early stages, and AMD will continue to explore tradeoffs against real-world benefits.
Much of the advantage of having an AI engine on board is power efficiency, which is a must for power constrained devices such as laptops, but not so much for unconstrained desktop PCs where inference workloads can use powerful dedicated GPUs or CPUs. It may not make sense. But don’t worry about battery life. He asked McAfee if these factors could influence AMD’s decision on whether to bring XDNA to desktop PCs, and he said that ultimately the feature would dedicate valuable die real estate. They answered that it would depend on whether they offered enough value to spend. that engine. AMD, in particular, is still evaluating the implications of his Ryzen 7040 launch.
For now, AMD has not confirmed anything about its future plans, but McAfee said that while AMD is working to include an AI engine as part of its future roadmap, it will be included in all of its products. He said it may not be the case. In that regard, he said there may be other options for various types of chips that take advantage of AMD’s chiplet strategy, such as desktop PCs. Other options such as add-in cards are also possible solutions.
One thing is certain: the scalable, integrated XDNA AI engine will continue to appear in many of AMD’s products. Hope to see a better demo next time.
