Samsung built the world’s first large-scale computing system using GPUs with embedded processing-in-memory (PIM) chips. Loaded with 96 AMD Instinct MI100 GPUs, these memory modules boosted AI training performance by 2.5x. Business Korea report.
Samsung originally demonstrated PIM-modified GPUs in October, but only recently combined 96 PIM-modified GPUs in a cluster. Compared to normal video memory, these modified MI100 chips not only improved performance by 2.5 times that of him, but also reduced power consumption by 2.67 times that of him, greatly increasing the efficiency of GPUs in running AI algorithms. Improved.
PIM is a new generation of computer memory that can accelerate complex computational workflows handled by processors such as CPUs and GPUs. As the name suggests, each memory module can handle data independently, reducing the amount of data that needs to be moved between memory and processor.
Samsung has been developing PIM for some time. The company has demonstrated several implementations in 2021, including several different memory types including DDR4, LPDDR5X, GDDR6, and HBM2. The LPDDR5 format showed a 42.6% reduction in power consumption, a 70% reduction in latency, and a 1.8x performance improvement in a test program that included the Meta AI workload. Even more impressive, these results were obtained from a standard server system with no changes to the motherboard or CPU (the only change being the replacement of his LPDDR5 DIMMs with PIM support. ).
Samsung is not the only company developing PIM chips. SK hynix released his own PIM module earlier this year. According to SK hynix preliminary tests, the company’s GDDR6-AiM (Accelerator in Memory) application sped up his AI processing by 16x and reduced power consumption by 80%. That’s much faster than Samsung’s modded MI100, but we don’t know what SK hynix used to test it, so it’s not a direct comparison.
Either way, PIM looks like a powerful solution for speeding up AI-accelerated workflows. “As the head of the AI research center, I want Samsung to become a semiconductor company that utilizes AI more than any other company,” said Choi Chang-gyu, vice president and director of the AI research center at Samsung Electronics Advanced Technology Research Institute. He told Business Korea.
