AI applications is driving the memory market forward, with HBM (High Bandwidth Memory) undoubtedly being a sought-after product of the industry, attracting increased capital expenditure and production expansion from memory manufacturers. At the meantime, a new force in the memory market has quietly emerged: GDDR7 is expected to drive the memory market steadily forward as HBM amid the AI wave.

• The Differences between GDDR7 and HBM

GDDR7 and HBM both belong to the category of graphics DRAM with high bandwidth and high-speed data transmission capabilities, providing strong support for AI computing. However, GDDR7 and HBM differ slightly in terms of technology, application scenarios, and performance.

GDDR7 is the latest technology in the GDDR family primarily used to enhance the available bandwidth and memory capacity of GPU. In March 2024, JEDEC, the Solid State Technology Association, officially released the JESD239 GDDR7 standard, which significantly increases bandwidth, eventually reaching 192GB/s per equipment.

It can be calculated that the memory speed is 48Gbps, double that of GDDR6X, the number of independent channels double from 2 in GDDR6 to 4 in GDDR7, and it supports densities ranging from 16-32 Gbit, including support for 2-channel mode to double system capacity.

Additionally, JESD239 GDDR7 is the first JEDEC-standard DRAM to use a Pulse Amplitude Modulation (PAM) interface for high-frequency operation. Its PAM3 interface improves the signal-to-noise ratio (SNR) in high-frequency operations while improving energy efficiency.

GDDR7 is mainly applied in graphics processing, gaming, computing, networking, and AI, particularly in gaming, where its high bandwidth and high-speed data transmission capabilities can significantly improve frame smoothness and loading speed, enabling a better experience for game players. In the field of AI, GDDR7 boasts great potential, capable of supporting rapid data processing and computation for large AI models, thus speeding up model training and inference.

Michael Litt, chairman of the JEDEC GDDR Task Group, has stated that GDDR7 is the first to focus not only on bandwidth but also on integrating the latest data integrity features to meet the market demands for RAS (Reliability, Availability, and Serviceability). These features allow GDDR devices to better serve existing markets like cloud gaming and computing, and expand its presence to AI sector.

Based on memory stacking technology, HBM connects layers through Through-Silicon Via (TSV), and features high capacity, high bandwidth, low latency, and low power consumption. Its strength lies in breaking the memory bandwidth and power consumption bottleneck. Currently, HBM is mainly used in AI server and supercomputer applications.

Since the introduction of the first generation in 2013, HBM has developed the second generation (HBM2), third generation (HBM2E), fourth generation (HBM3), and fifth generation (HBM3E).

This year, HBM3e will be the mainstream in the market, with concentrated shipments expected in 2H24. Besides, the sixth generation HBM4 is anticipated to make its debut as early as 2025. Reportedly, HBM4 will bring revolutionary changes, adopting a 2048-bit memory interface, which theoretically can double the transmission speed again.

• Three Memory Giants Scramble for the Initiative in GDDR7 Market

Due to high technical barriers, HBM market share is firmly at the helm of the three major memory players: SK Hynix, Samsung, and Micron. With the ongoing influence of AI, their competition has been expanding from HBM to GDDR field.

Since the beginning of this year, the three manufacturers have successively announced the availability of GDDR7 memory samples. It’s expected that some of them will start mass production of GDDR7 between 4Q24 and 1Q25.

Photo credit: Samsung Electronics

In March, Samsung and SK Hynix announced their respective GDDR7 specifications. Samsung’s GDDR7 chip, using PAM3 signal for the first time, can achieve a speed of 32Gbps at a DRAM voltage of only 1.1V, exceeding the JEDEC GDDR7 specification of 1.2V.

SK Hynix’s latest GDDR7 product, compared to its predecessor GDDR6, offers a maximum bandwidth of 160GB/s, double that of the previous generation, with a 40% improvement in power efficiency and a 1.5 times increase in memory density.

In June, Micron announced it already begun sampling its new generation of GDDR7, achieving a speed of 32Gbps and a memory bandwidth of 1.5TB/sec, a 60% improvement over GDDR6, boasting the industry’s highest bit density. Micron’s GDDR7 utilizes 1β DRAM technology and an innovative architecture and has four independent channels to optimize workloads, offering faster response time, smoother gaming experience, and shorter processing time.

Additionally, Micron’s GDDR7 improves energy efficiency by 50% relative to GDDR6, which hence enhances thermal performance for portable devices (Like laptop) and extends battery lifespan. The new sleep mode can reduce standby power consumption by 70%. Micron claims its next-generation GDDR7 can deliver high performance, increasing throughput by 33% and reducing response time for generative AI workloads (Text and image creation included) by 20%.

Photo credit: Micron

Recently, rumor has it that NVIDIA RTX 50 series will fully adopt the latest GDDR7, with a maximum capacity of 16GB, including models GN22-X11 (16 GB GDDR7), GN22-X9 (16 GB GDDR7), GN22-X7 (12 GB GDDR7), GN22-X6 (8 GB GDDR7), GN22-X4 (8 GB GDDR7), and GN22-X2 (8 GB GDDR7). The industry believes that GDDR7 will become a new arena in the memory market following HBM, in which manufacturers will continue to battle for NVIDIA GPU orders.

Read more

• [News] SK hynix to Kick off Mass Production for GDDR7 in Q4 2024
• [News] Micron Reportedly Targets 25% HBM Market Share by 2025

(Photo credit: Samsung Electronics)

Driven by memory giants ramping up high-bandwidth memory (HBM) production, according to a report from Korean media outlet TheElec, ASMPT, a back-end equipment maker, has supplied a demo thermal compression (TC) bonder for Micron’s HBM production.

TC bonders play a pivotal role in HBM production by employing thermal compression to bond and stack chips on processed wafers, thereby significantly influencing HBM yield.

ASMPT is reportedly collaborating with the US memory giant to co-develop a TC bonder for use in HBM4 production. Notably, ASMPT has supplied TC bonders to SK Hynix as well and plans to deliver more units later in the year.

Micron is also procuring TC bonders from Shinkawa and Hanmi Semiconductor for the production of HBM3e. However, as per the same report citing sources, Shinkawa has its handful in supplying the bonders to its largest customer, so Micron added Hanmi Semiconductor as a secondary supplier.

In addition to Micron, Samsung Electronics and SK Hynix have developed distinct supply chains for TC bonders. Samsung sources its equipment from Japan’s Toray and Sinkawa, as well as its subsidiary SEMES. In contrast, SK Hynix relies on Singapore’s ASMPT, HANMI Semiconductor, and Hanhwa Precision Machinery.

According to industry sources cited by The Chosun Daily, TC bonder orders driven by memory giants have been strong, as Samsung Electronics’ subsidiary SEMES has delivered nearly 100 TC bonders over the past year. Meanwhile, SK Hynix has inked a approximately $107.98 million contract with HANMI Semiconductor, which commands a 65% share of the TC bonder market.

Regarding the latest developments in HBM, TrendForce indicates that HBM3e will become the market mainstream this year, with shipments concentrated in the second half of the year. Currently, SK hynix remains the primary supplier, along with Micron, both utilizing 1beta nm processes and already shipping to NVIDIA.

According to TrendForce predictions, the annual growth rate of HBM demand will approach 200% in 2024 and is expected to double in 2025.

Read more

• [News] Rising HBM Production by Samsung and SK Hynix Energizes Korea’s TC Bonder Industry
• [News] Micron’s HBM Expansion at Full Throttle: Rumored to Expand in US, Malaysia Also an Option

(Photo credit: Micron)

According to a report by Korean media outlet Business Korea, SK Hynix recently shared its latest breakthrough on its 3D DRAM at VLSI 2024 last week, announcing that the manufacturing yield of its 5-layer stacked 3D DRAM has reached 56.1%.

This means that out of roughly 1,000 3D DRAM units manufactured on a single test wafer, about 561 functional devices were successfully manufactured, the report further explains. The experimental 3D DRAM exhibits characteristics similar to the currently used 2D DRAM, marking the first time SK Hynix has disclosed specific numbers and characteristics of its 3D DRAM development.

However, SK Hynix also noted that while 3D DRAM holds great potential, a significant amount of development is still required before it can be commercialized. The memory giant also reportedly pointed out that unlike the stable operation of 2D DRAM, 3D DRAM exhibits unstable performance characteristics, and stacking 32 to 192 layers of memory cells is necessary for widespread use.

3D DRAM is also a key development area for other major memory manufacturers like Samsung Electronics and Micron. Samsung Electronics has successfully stacked 3D DRAM up to 16 layers and plans to mass-produce 3D DRAM around 2030. Micron currently holds 30 patents related to 3D DRAM, and if there are breakthroughs in 3D DRAM technology, it could produce better DRAM products than existing ones without the need for EUV equipment.

The DRAM market remains highly concentrated, currently dominated by key players such as Samsung Electronics, SK Hynix, and Micron Technology, collectively holding over 96% of the entire market share.

Read more

• [News] Samsung and SK hynix to Implement Hybrid Bonding with 3D DRAM
• [News] The Quiet Beginning of the 3D DRAM Market Share Battle

(Photo credit: SK Hynix)

According to a report by the Economic Daily News, TSMC has secured another AI business opportunity. Following its exclusive contract manufacturing of AI chips for tech giants such as NVIDIA and AMD, TSMC, in collaboration with its subsidiary, the ASIC design service provider Global Unichip Corporation (GUC), has reportedly made significant progress in producing essential peripheral components for AI servers, specifically high-bandwidth memory (HBM). Together, they have secured a major order for the foundational base die chips of next-generation HBM4.

TSMC and GUC typically do not comment on order details. SK Hynix, on the other hand,  has clarified in a press release to Bloomberg that it has not signed a contract with GUC for its next-generation AI memory chips, according to the Economic Daily News.

Industry sources cited by the report point out that the strong demand for AI is not only making high-performance computing (HPC) related chips highly sought after, but also driving robust demand for HBM, creating new market opportunities. This surge in demand has attracted major memory manufacturers such as SK Hynix, Samsung, and Micron to actively invest. Under the influence of AI engines, the current production capacity for HBM3 and HBM3e is in a state of supply shortage.

As AI chip manufacturing advances to the 3nm generation next year, the existing HBM3 and HBM3e, limited by capacity and speed constraints, may prevent the new generation of AI chips from reaching their maximum computational power. Consequently, the three major memory manufacturers are unanimously increasing their capital expenditures and starting to invest in the development of next-generation HBM4 products, aiming for mass production by the end of 2025 and large-scale shipments by 2026.

While memory manufacturers are delving into the research and development of next-generation HBM4, the semiconductor standardization organization JEDEC Solid State Technology Association is also busy establishing new standards related to HBM4. It’s also rumored that JEDEC will relax the stacking height limit for HBM4 to 775 micrometers, hinting that the previously required advanced packaging technology using hybrid bonding can be postponed until the next generation of HBM specifications.

Industry sources cited by the report also suggest that the most significant change in HBM4, besides increasing the stacking height to 16 layers of DRAM, will be the addition of a logic IC at the base to enhance bandwidth transmission speed. This logic IC, known as the base die, is expected to be the major innovation in the new generation of HBM4 and possibly a reason for JEDEC’s relaxation of the stacking height limitation.

On the other hand, SK Hynix has announced its collaboration with TSMC to advance HBM4 and capture opportunities in advanced packaging. Industry sources also indicate that GUC has successfully secured the critical design order for SK Hynix’s HBM4 base die.

The design is expected to be finalized as early as next year, with production to be carried out using TSMC’s 12nm and 5nm processes, depending on whether high performance or low power consumption is prioritized.

Reportedly, it’s suggested that SK Hynix’s decision to entrust the base die chip orders to GUC and TSMC is primarily because TSMC currently dominates over 90% of the CoWoS advanced packaging market used in HPC chips.

Read more

• [News] SK Hynix Disclosed Details Regarding HBM4e, Reportedly Integrating Computing and Caching Functionalities
• [News] SK Hynix Partners with TSMC to Strengthen HBM Technological Leadership

(Photo credit: TSMC)

SK keyfoundry, a subsidiary of memory giant SK hynix, has achieved notable progress in the development of Gallium Nitride (GaN) power semiconductors. According to the latest report by Business Korea, the foundry would begin producing power semiconductors for Tesla in the second half of 2024.

According to the report, SK keyfoundry announced in early June that it has achieved the primary device characteristics of a 650V GaN High Electron Mobility Transistor (HEMT), which surpasses traditional silicon-based semiconductors in both efficiency and durability. This advancement aligns with SK keyfoundry’s plan to finalize the development of GaN power semiconductors by the end of this year.

It is worth noting that TSMC has also entered the GaN market a few years ago, as it provides GaN process for manufacturing 100/650V discrete GaN power devices for customers. For instance, in 2020, the world’s largest foundry has announced to collaborate with STMicroelectronics. According to its press release, ST’s GaN products will be manufactured using TSMC’s leading GaN process technology, including applications relating to automotive converters and chargers for hybrid and electric vehicles.

Regarding the development of SK keyfoundry, Business Korea noted that the company established an official team in 2022 to focus on the development of GaN technologies. Citing industry sources on June 20^th, the report stated that SK keyfoundry will reportedly begin producing power semiconductors for Tesla in the second half of this year.

Moreover, it also mulls to broaden its business scope, entering markets like fast-charging adapters, data centers, and energy storage systems afterwards. Starting in November, the company plans to manufacture power management chips (PMIC) at its 8-inch wafer fab in Cheongju.

Though foundries have not significantly contributed to SK hynix’s revenue so far, the development of power semiconductors could boost overall foundry sales. According to the report, SK keyfoundry also provides contract manufacturing for non-memory semiconductors such as Display Driver ICs (DDI) and Microcontroller Units (MCU), further diversifying its product lineup.

In the current landscape of the new energy market, third-generation semiconductors such as SiC and GaN have gained significant traction. SiC (Silicon Carbide) and GaN could offer significant benefits over traditional silicon.

To elaborate, semiconductor materials have the so-called “bandgap,” an energy range in a solid where no electrons can exist. According to German chipmaker Infineon, GaN has a bandgap of 3.4 eV, compared to silicon’s 1.12 eV bandgap. The wider bandgap of GaN allows it to sustain higher voltages and temperatures than silicon. While SiC dominates the high-power domain, GaN excels at lower power levels, offering lower conduction losses.

Read more

• [News] Infineon in GaN and SiC Expansion Drive Advancements in the New Energy Market

(Photo credit: SK keyfoundry)