HBM3 Memory Manufacturing Is a Critical Enabler of Generative AI (Guest Blog)

Sometimes the technology behind the scenes is just as impressive as the one that gets all the attention.

So it is with Generative AI. While most everyone seems to be falling over themselves proclaiming how revolutionary GenAI is and will be, there's been little attention focused on some of the key enabling technologies that power this industry-changing trend. Admittedly, GPUs have certainly gained more notoriety thanks to the critical role these chips play in accelerating the performance of the foundation models that underlay today's GenAI-powered applications and services.

But a huge part of what GPUs are able to achieve is because of a relatively new form of high-speed memory that allows huge data sets to be fed into the chip at record speeds. Known colloquially via their acronym, HBM, these High Bandwidth Memory chips play an important but less understood role in today's GenAI workloads. The third major generation of these chips (hence the name HBM3), in particular, have proven to be very good at keeping the latest generation GPU architectures as busy and as efficient as possible by providing super high-speed access (up to 6.4 Gb/s) to the vast data sets used for both AI training and inferencing workloads.

What's unique about HBM3 is that it provides a multilayer stack of DRAM (Dynamic Random Access Memory) chips in a single package. So, instead of having single-layer DRAMs, as older designs used to do, HBM3 stacks up to 12 layers of memory chips on top of each other, reducing the space and power consumption of individual chips while simultaneously speeding access to larger amounts of memory. Frankly, it's a great solution that brings together multiple benefits in a clever way.

But what is really exciting, it turns out, is in how you manufacture HBM chips. Advanced packaging technologies, such as 2.5D and 3D integration, enable the stacking of memory and logic chips. And that's where Lam Research and its semiconductor manufacturing equipment come into play, specifically its industry-leading Syndion® etch and SABRE 3D® deposition equipment.

The way memory chips work is that they store a small charge within what's called a memory cell of the chip. Well actually, they store billions of small charges within billions of tiny memory cells. When a charge is present in the memory cell, that represents a 1 in the binary world of computer data. If no charge is present, that represents a 0.

Building single-layer versions of these chips is hard enough, given the massive number of memory cells they include, but moving to stackable HBM3 designs adds several other critical challenges. Most notable is the interconnect between the different layers of the stack, which is what allows the memory to function as a high-speed contiguous unit. These connections are made by something called Through Silicon Vias, or TSVs, which provide the physical and electrical connections between the layers. Physically, they're very tall structures that act as the wiring connections between the stacked chips. Functionally, they are critical technology for advanced semiconductor packaging, which is what is allowing both more complex and higher performing semiconductor parts to now be manufactured.

The theory of what TSVs are and what they do sounds relatively simple but manufacturing them in the real-world is an extraordinary technical challenge from both a physics and chemistry perspective. First, you have to create the "holes," ensuring that they're perfectly round, symmetrical and aligned all the way through. Then you have to apply several thin layers of material as a "coating" to those holes. Finally, you fill the holes with tiny amounts of metal. Oh, and you have to do millions of these TSVs with incredible precision.

Each of these three steps is performed by different types of machines. The initial holes are created with etching equipment that, as its name suggests, etches out some of the raw silicon material on a memory chip. The second step involves three layers of deposition, all done at an atomic scale. The first layer can be done by a rapid atomic layer deposition (ALD) tool. Materials are deposited in the holes to create the remaining two layers of coating through advanced physical sputtering, which may involve bombarding the source material with ions. Finally, the aptly named electrofill tool provides the metal to fill those holes and create tiny microbump connectors.

Because up to 12 (and soon to be 16) layers of these chips need to be stacked on top of each other and each of the TSVs from one layer have to align with the TSVs from the next layer, the precision and consistency of all those steps have to meet extraordinarily tight tolerances. Lam's Syndion etching equipment uses deep reactive ion etching to create the kinds of deep holes required for TSVs. Its Striker® ALD product offers the ability to produce film coating layers as thin as a few atoms - or about 1/10,000 the width of a human hair. And its SABRE 3D copper electroplating tool is specifically designed for the advanced packaging applications that TSVs represent.

Each of these semiconductor manufacturing equipment tools offer the kind of accuracy, uniformity and performance needed to drive high-volume production of many different types of memory chips and other semiconductor devices. More importantly, collectively, the three create an extremely powerful system of tools for creating TSVs. Building these TSVs in a reliable and high-speed manner can help memory makers meet the nearly insatiable demand for HBM that currently exists and likely will well into the future.

There's no doubt that GenAI is going to have a profound impact not only on the tech industry, but industries of all kinds. The kinds of applications it's enabling and the impact it's having on how we interact with computing devices will eventually have a dramatic impact on virtually all our lives in some way or another. But as important as GenAI is, it's also critical to remember how essential the underlying silicon technology that's enabling it is as well. That's where things like GPUs, HBM3 memory and more can make a significant difference too. Knowing how these elements are manufactured, and the tools required to make that happen, is a critical part of the bigger story.

Bob O'Donnell is the president and chief analyst of TECHnalysis Research, LLC a market research firm that provides strategic consulting and market research services to the technology industry and professional financial community. You can follow him on Twitter @bobodtech.

Public link

Please use the above public link if you want to share this noodl on another website.