The AI Memory Moat

SK Hynix is the undisputed leader in High Bandwith Memory

Just behind the AI processing battle is another equally exciting race: High Bandwidth memory competition between the three large DRAM companies, Samsung, Micron and SK Hynix.

SK Hynix has been the leader so far. The Korean memory company is well-known for being the leading supplier of HBM to Nvidia’s H100 GPU. As SK Hynix recently announced the Q3-24 result, there is fresh data to analyse that can help us understand the HBM situation and the difference between the players.

The recent pre-result from Samsung suggests that the HBM moat is still wide, but we will find out later this week when Samsung reports fully.

No AI without High Bandwidth Memory

A key element of the accelerated computing revolution that has brought us AI is the use of highly parallel compute engines (GPUs and accelerators) that need a large amount of memory that can be accessed at high speeds.

To get the needed performance increase, memory needs to be as close to the GPU as possible, which can only be done with the highly dense and parallel architecture of the HMBs and their interfaces. This is done using advanced packaging methods currently led by TSMC, with Intel and Samsung chasing.

The technology is very advanced and sensitive. DRAM does not like heat and the GPU is hot as an oven. (Read more in Cooling the AI Furnace)

“DRAM hates heat. It starts to forget stuff about 85°C, and is fully absent-minded about 125°C.”

Marc Greenberg, Group Director, Cadence

From a technological perspective, HBMs are much more advanced than GPUs that excel in architecture and software. (Read more inThe Light that burns twice as bright)

Drilling into the SK Hynix Q3-24 result

Another step away from the worst semiconductor cycle abyss, Hynix could deliver a great result with record revenue, operating profits and net profit. The company’s press release could not hide the excitement of SK Hynix cementing its position as the leading memory company while Samsung apologises to its stakeholders.

The revenue was up 8.2% QoQ and 88% YoY, while gross profit and operating profits grew 23.7% and 30%, respectively, QoQ.

The discrepancy between revenue and profit growth results from the increasing share of HBM, which grew to 80% QoQ. The HBM share of DRAM is now 30%, and SK Hynix expects it to increase to 40% in Q4.

While this will positively impact the Q4-24 result, the HBM business is long-term, and pricing has already been negotiated and settled several quarters ahead. The gap between standard memory and HBM will diminish as we get closer to the peak, and in all likelihood, standard DRAM will have a higher gross margin than HBM in the peak quarters.

The inventory situation reveals more about the situation in the machine room. The total inventory has declined slightly since the last bottom was passed.

The central part of the inventory that has declined is the finished goods inventory, while the work-in-progress has stayed relatively stable. This indicates that manufacturing is stable and currently at its maximum level.
The 8 BW level equates to 6B$, slightly below the quarterly COGS, indicating a stable production cycle of around a quarter.

Part of the increase in inventory is the shift to HBM, which can be seen in the COGS chart. HBM are both more expensive to produce, and the cycle time is longer. But as seen in the WIP inventory, SK Hynix is now fully transformed into an HBM-focused company that will max out at a level above the 40% revenue target for next quarter.

The reason that the production is maxed out is uncovered by investigating SK Hynix Property, Plant and Equipment position.

In the last 6 quarters, the value of SK Hynix fixed assets has been declining by more than 10%. This is a clear sign of underinvestment well below maintenance investment. In other words, the SK Hynix capacity declined during the same period.

This results from the worst memory downcycle I can remember, and it is understandable when running at a Gross profit below 32%. The memory market is not for the faint at heart.

It is worth analysing SK Hynix and its two main competitors to better understand the investment level.

The competitive market context

The three leading memory companies all have to abide by the cruel semiconductor cycle that requires massive investment decisions at the most challenging periods.

As can be seen, Samsung is the only company that grew its Memory capacity in terms of PPE during the last downcycle in opposition to Micron and SK Hynix.

The two companies scaled back on CapEx, while Samsung continued to invest more than the two others combined. While it takes time for CapEx and PPE to become extra capacity, this was a bold move.