When SK hynix first announced its HBM3 memory portfolio in late 2021, the company said it was developing both an 8-Hi 16GB memory stack and a more technically complex 12-Hi 24GB memory stack. . Now, almost 18 months after its initial announcement, SK hynix has finally started offering samples of its 24GB HBM3 stack to multiple customers. These are all very welcome developments for SK hynix’s downstream customers, many of whom require additional memory capacity to meet the needs of large language models and other high-end computing uses. is.
Based on the same technology as SK hynix’s existing 16GB HBM3 memory module, the 24GB stack is designed to further increase the overall HBM3 memory module density by increasing the number of DRAM layers from 8 to 12. % more capacity. This has been in the HBM specification for quite some time, but has proven difficult to implement as it requires the very thin DRAM dies in the stack to be made thinner.
Standard HBM DRAM packages are typically 700-800 microns High (Samsung claims their 8-Hi and 12-Hi HBM2E are 720 microns tall). Ideally, the height should be maintained for these higher density stacks to be physically compatible with existing product designs. Avoid towering over processors with which they are paired. As a result, to squeeze 12 memory devices into a standard KGSD, memory manufacturers must either shrink the thickness of each DRAM layer, shrink the space between layers, or shrink the space between layers without sacrificing performance or yield. Base layers should be minimized or a combination of all three measures should be deployed. .
Details are limited in SK hynix’s latest press release, but it looks like they are trying to thin the DRAM dies and the space between them with an improved underfill material. As for the DRAM die itself, SK hynix previously said it was able to shave the die thickness down to 30 microns. Meanwhile, an improved underflow material for 12-Hi stacks is offered as part of the company’s new offerings. Mass reflow molded underfill (MR-MUF) packing technology. In this technology, the DRAM dies are bonded together by a reflow process, while filling the gaps between the dies with an underfill material.
SK hynix calls the improved underfill material “liquid epoxy molding compound” or “liquid EMC” and it replaces the old non-conductive film (NCF) used in previous generations of HBM. Of particular interest here is that in addition to the thinner layers this allows, it also offers twice the thermal conductivity of NCF according to SK hynix Liquid EMC. Keeping the bottom layer of stacked chips reasonably cool has been one of the biggest challenges of any kind of chip stacking technology, he said. So doubling the thermal conductivity of the filler material is a big improvement for SK hynix. Improving heat dissipation from the well-embedded bottom die should go a long way towards making 12-Hi stacks more viable.
Assembly aside, SK hynix’s 24GB HBM3 stack has the same performance specs as the existing 16GB stack. This means that the maximum data transfer rate running on the 1024-bit interface is 6.4Gbps/pin, providing a total bandwidth of 819.2 GB/s per stack.
Ultimately, any assembly difficulties with a 12-Hi HBM3 stack should be justified by the benefits the additional memory capacity brings. A major SK hynix customer has already adopted six or more of his HBM3 stacks in one product to provide the total bandwidth and memory capacity they deem necessary. A 50% increase in memory capacity will greatly benefit products such as GPUs and other forms of AI accelerators. Especially in this age of large language models, memory capacity has become a bottleneck for model training. NVIDIA is pushing the limits of memory capacity with the H100 NVL (a special 96 GB H100 SKU that enables previously reserved memory) so 24 GB HBM3 can be used to offer 120 GB/144 GB H100 parts It is easy to understand that you are eager to do so. Stack.
sauce: SK Hynix
