Ahead of SK hynix’s Q3 earnings announcement on October 24^th, the market expects it may see a surge in quarterly operating profit driven by HBM, potentially leading the company to outperform Samsung’s semiconductor division. Therefore, there is growing interest in SK hynix’s next move.

In order to maintain its leadership in the memory sector amid heated competition from China, SK hynix is reportedly shifting its focus to high-value technologies such as HBM4 and Compute Express Link (CXL), according to Korean media outlet Pinpoint News and Tom’s Hardware.

This shift is motivated by a highly competitive memory market, where Chinese firms are ramping up their production capabilities and adopting aggressive pricing strategies to gain share, Tom’s Hardware notes.

For instance, a previous report by ZDNet mentions that Chinese memory manufacturers like CXMT (Changxin Memory Technologies) are aggressively expanding production, which could negatively affect profitability in the traditional DRAM market. Established in 2016, CXMT has become China’s largest DRAM producer with government backing.

Both Samsung and SK hynix are said to be closely monitoring these developments, and counting on high-valued technologies like HBM4 and CXL to unlock a new wave of growth momentum.

It is worth noting that both memory giants have teamed up with TSMC on HBM4, as they attempt to incorporate customized features requested by major clients, counting on TSMC to manufacture more powerful logic dies, the component that functions as the brain of an HBM chip.

Per SK hynix’s product roadmap, the company plans to launch 12-layer stacked HBM4 in the second half of 2025 and 16-layer in 2026.

Samsung, which is struggling with the 12-Hi HBM3e verification with NVIDIA, also aims high for HBM4 to turn the tide. A previous report by The Elec indicates that Samsung targets to tape out HBM4 by year-end, while eyeing the mass production by the end of 2025.

On the other hand, CXL is a next-generation interface that efficiently connects CPUs, GPUs, and memory in high-performance computing systems. According to Pinpoint News, by applying CXL to existing memory modules, capacities can be expanded by more than ten times, makes it extremely suited for the demand of the AI era.

SK hynix is also focusing on CXL memory, which is gaining attention as the next-generation AI memory following HBM. Citing SK hynix CEO Kwak Noh-Jung’s remarks, a report by ZDNet suggests that the memory giant plans to launch products like CXL and LPCAMM tailored to customers’ needs, as the results will begin to materialize around next year.

In the meantime, Samsung reportedly aims to begin mass production of a 256GB CMM-D, compatible with the CXL 2.0 protocol, by the end of 2024, according to Tom’s Hardware. In its own words, Samsung’s CMM-D is a memory expander built with next-generation CXL technology, which seamlessly connects multiple processors and devices, increasing memory capacity thus optimizing memory management.

Read more

• [News] Samsung’s Next Move? Early HBM4 Mass-production and 2nm Foundry Solutions Could Be the Cure
• [News] SK hynix Reportedly to Tape-out HBM4 in October, Paving the Way for NVIDIA’s Rubin

(Photo credit: SK hynix)

According to a report from Nikkei, Samsung Electronics, currently lagging behind SK hynix in the HBM market, is said to be betting on the next-generation CXL memory, with shipments expected to begin in the second half of this year, while anticipating the CXL memory to become the next rising star in AI.

CXL is a cache-coherent interconnect for memory expansion, which may maintain memory coherency between the CPU memory space and memory on attached devices, which allows resource sharing for higher performance.

The CXL module stacks DRAM layers and connects different semiconductor devices like GPUs and CPUs, expanding server memory capacity up to tenfold.

Choi Jang-seok, head of Samsung Electronics’ memory division, explained that CXL technology is comparable to merging wide roads, enabling the efficient transfer of large volumes of data.

As tech companies rush to develop AI models, existing data centers are gradually becoming unable to handle the enormous data processing demands.

As a result, companies are beginning to build larger-scale data centers, but this also significantly increases power consumption. On average, the energy required for a general AI to answer user queries is about ten times that of a traditional Google search.

Choi further highlighted that incorporating CXL technology allows for server expansion without the need for physical growth.

In 2021, Samsung became one of the first companies in the world to invest in the development of CXL. This June, Samsung announced that its CXL infrastructure had received certification from Red Hat.

Additionally, Samsung is a member of the CXL Consortium, which is composed of 15 tech companies, with Samsung being the only memory manufacturer among them. This positions Samsung to potentially gain an advantage in the CXL market.

While HBM remains the mainstream memory used in AI chipsets today, Choi Jang-seok anticipates that the CXL market will take off starting in 2027.

Since the surge in demand for NVIDIA’s AI chips, the HBM market has rapidly expanded. SK hynix, which was the first to develop HBM in 2013, has since secured the majority of NVIDIA’s orders, while Samsung has lagged in HBM technology.

Seeing Samsung’s bet on CXL, SK Group Chairman Chey Tae-won remarked that SK Hynix should not settle for the status quo and immediately start seriously considering the next generation of profit models.

Read more

• News] TSMC Reportedly Invests over USD 12.3 Billion in EUV, Advancing in 2nm Production
• [News] Intel Secures First Batch of High-NA EUV Equipment from ASML, Ahead of Samsung and SK Hynix

(Photo credit: Samsung)

According to TrendForce’s latest report on the server industry, not only have emerging applications in recent years accelerated the pace of AI and HPC development, but the complexity of models built from machine learning applications and inferences that involve increasingly sophisticated calculations has also undergone a corresponding growth as well, resulting in more data to be processed. While users are confronted with an ever-growing volume of data along with constraints placed by existing hardware, they must make tradeoffs among performance, memory capacity, latency, and cost. HBM (High Bandwidth Memory) and CXL (Compute Express Link) have thus emerged in response to the aforementioned conundrum. In terms of functionality, HBM is a new type of DRAM that addresses more diverse and complex computational needs via its high I/O speeds, whereas CXL is an interconnect standard that allows different processors, or xPUs, to more easily share the same memory resources.

HBM breaks through bandwidth limitations of traditional DRAM solutions through vertical stacking of DRAM dies

Memory suppliers developed HBM in order to be free from the previous bandwidth constraints posed by traditional memory solutions. Regarding memory architecture, HBM consists of a base logic die with DRAM dies vertically stacked on top of the logic die. The 3D-stacked DRAM dies are interconnected with TSV and microbumps, thereby enabling HBM’s high-bandwidth design. The mainstream HBM memory stacks involve four or eight DRAM die layers, which are referred to as “4-hi” or “8-hi”, respectively. Notably, the latest HBM product currently in mass production is HBM2e. This generation of HBM contains four or eight layers of 16Gb DRAM dies, resulting in a memory capacity of 8GB or 16GB per single HBM stack, respectively, with a bandwidth of 410-460GB/s. Samples of the next generation of HBM products, named HBM3, have already been submitted to relevant organizations for validation, and these products will likely enter mass production in 2022.

TrendForce’s investigations indicate that HBM comprises less than 1% of total DRAM bit demand for 2021 primarily because of two reasons. First, the vast majority of consumer applications have yet to adopt HBM due to cost considerations. Second, the server industry allocates less than 1% of its hardware to AI applications; more specifically, servers that are equipped with AI accelerators account for less than 1% of all servers currently in use, not to mention the fact that most AI accelerators still use GDDR5(x) and GDDR6 memories, as opposed to HBM, to support their data processing needs.

Although HBM currently remains in the developmental phase, as applications become increasingly reliant on AI usage (more precise AI needs to be supported by more complex models), computing hardware will then require the integration of HBM to operate these applications effectively. In particular, FPGA and ASIC represent the two hardware categories that are most closely related to AI development, with Intel’s Stratix and Agilex-M as well as Xilinx’s Versal HBM being examples of FPGA with onboard HBM. Regarding ASIC, on the other hand, most CSPs are gradually adopting their own self-designed ASICs, such Google’s TPU, Tencent’s Enflame DTU, and Baidu’s Kunlun – all of which are equipped with HBM – for AI deployments. In addition, Intel will also release a high-end version of its Sapphire Rapids server CPU equipped with HBM by the end of 2022. Taking these developments into account, TrendForce believes that an increasing number of HBM applications will emerge going forward due to HBM’s critical role in overcoming hardware-related bottlenecks in AI development.

A new memory standard born out of demand from high-speed computing, CXL will be more effective in integrating resources of whole system

Evolved from PCIe Gen5, CXL is a memory standard that provides high-speed and low-latency interconnections between the CPU and other accelerators such as the GPU and FPGA. It enables memory virtualization so that different devices can share the same memory pool, thereby raising the performance of a whole computer system while reducing its cost. Hence, CXL can effectively deal with the heavy workloads related to AI and HPC applications.

CXL is just one of several interconnection technologies that feature memory sharing. Other examples that are also in the market include NVLink from NVIDIA and Gen-Z from AMD and Xilinx. Their existence is an indication that the major ICT vendors are increasingly attentive to the integration of various resources within a computer system. TrendForce currently believes that CXL will come out on top in the competition mainly because it is introduced and promoted by Intel, which has an enormous advantage with respect to the market share for CPUs. With Intel’s support in the area of processors, CXL advocates and hardware providers that back the standard will be effective in organizing themselves into a supply chain for the related solutions. The major ICT companies that have in turn joined the CXL Consortium include AMD, ARM, NVIDIA, Google, Microsoft, Facebook (Meta), Alibaba, and Dell. All in all, CXL appears to be the most favored among memory protocols.

The consolidation of memory resources among the CPU and other devices can reduce communication latency and boost the computing performance needed for AI and HPC applications. For this reason, Intel will provide CXL support for its next-generation server CPU Sapphire Rapids. Likewise, memory suppliers have also incorporated CXL support into their respective product roadmaps. Samsung has announced that it will be launching CXL-supported DDR5 DRAM modules that will further expand server memory capacity so as to meet the enormous resource demand of AI computing. There is also a chance that CXL support will be extended to NAND Flash solutions in the future, thus benefiting the development of both types of memory products.

Synergy between HBM and CXL will contribute significantly to AI development; their visibility will increase across different applications starting in 2023

TrendForce believes that the market penetration rate of CXL will rise going forward as this interface standard is built into more and more CPUs. Also, the combination of HBM and CXL will be increasingly visible in the future hardware designs of AI servers. In the case of HBM, it will contribute to a further ramp-up of data processing speed by increasing the memory bandwidth of the CPU or the accelerator. As for CXL, it will enable high-speed interconnections among CPU and other devices. By working together, HBM and CXL will raise computing power and thereby expedite the development of AI applications.

The latest advances in memory pooling and sharing will help overcome the current hardware bottlenecks in the designs of different AI models and continue the trend of more sophisticated architectures. TrendForce anticipates that the adoption rate of CXL-supported Sapphire Rapids processors will reach a certain level, and memory suppliers will also have put their HBM3 products and their CXL-supported DRAM and SSD products into mass production. Hence, examples of HBM-CXL synergy in different applications will become increasingly visible from 2023 onward.

For more information on reports and market data from TrendForce’s Department of Semiconductor Research, please click here, or email Ms. Latte Chung from the Sales Department at lattechung@trendforce.com