The surge in AIGC and new technologies such IoT, AI, 5G, AR/VR are driving a huge demand for computational power of high-end chips. This has been even outpacing the performance increase offered by the long-standing Moore’s Law, ushering in a “post-Moore” era where revolutions in advanced chip design are crucial.

Over recent years, chiplet design has seemingly become the mainstream approach for upgrading high-end chips. The concept is to allow more transistors on a single chip, effectively increasing the production yield of high-end chips while reducing overall costs.

By the large, major IC players have all jumped on board. Even Apple has joined the game by releasing their M1 Ultra SoC using the chiplet concept, which doubles computational performance by integrating two M1 Max units in a single chip.

The CPU sector is definitely a clear demonstration of this trend:

• AMD took the leap with chiplet design in their 2nd-gen EPYC CPUs, doubling the computing cores from 32 to 64 within two years, while slashing costs by up to half. The company has extended this approach to their 4th-gen EPYC CPUs and even pioneered the GPU Navi 31, the first of its kind to use chiplets.
• Intel started incorporating chiplets into their Lakefield series SoC in 2020. Looking ahead, their upcoming CPUs like the Meteor Lake set for 2023, and Arrow Lake and Lunar Lake scheduled for 2024, will all use chiplet design.

Transition from Bumping to Hybrid Bonding

Our analysis in “Chiplet Design: A Real Game-Changer for Substrates” laid out the comprehensive impact of the evolution of chiplet technology on substrates. In fact, chiplets have already caused a significant disruption to the most advanced semiconductor packaging technologies, necessitating the transition towards advanced 2.5D and 3D packaging technologies.

The bottleneck of advanced packaging lies in the chiplets’ interconnections, with bump and microbump still being the key technology for linking chips and forming I/O joints. These connection densities are hard to enhance, thus limiting the overall chip’s transmission speed. In addition, the more chiplets being stacked, the bigger the chip volume gets. The challenge is how to limit the chip size within a specific range, considering the current technical constraints.

Therefore, copper-to-copper hybrid bonding, also known as DBI (Direct Bond Interconnect), has been emerging as the key technology route that overcomes major hurdles in chiplet integration from the bottom-up.

Unlike bumping technology, hybrid bonding significantly shrinks the I/O joint space. The future transmission demand requires the I/O joint space between chiplets to be less than 10µm. While bumping is limited to around 20µm, hybrid bonding can take this down to an impressive 1µm or even less. This also means more I/O joints can be fitted in the same chip size – even reaching up to millions on a mere 1cm2 chip.

On top of this, hybrid bonding only adds an extra 1-2µm of thickness, compared to the 10-30µm of microbump, thereby helping reduce the thickness of stacked chips.

To put it simply, hybrid bonding can boost transmission efficiency, minimize energy usage with higher density of copper joints, manage chip volume, and even cut down on material costs.

The Race for Advanced Packaging Is Kicking Off

Moving forward, hybrid bonding is set to become the key technology supporting the continuous development of chiplet design and 3D packaging. This has been exemplified by TSMC’s front-end So IC packaging technology which is based on hybrid bonding. This puts AMD, a key customer of TSMC, in a favorable position to get ahead.

From AMD’s roadmap of 3D V-Cache technology, they have stacked SRAM on top of CCX (CPU Complex), and gradually integrated it into Milan-X series, the EPYC server CPUs, and Ryzen series, the consumer-grade CPUs, over the past two years. This has significantly improved performance and power consumption as a whole.

Not to be outdone, this year Intel also launched their Foveros Direct packaging technology, which is also based on hybrid bonding route. Assuming everything proceeds smoothly, we can anticipate the release of CPUs utilizing Foveros Direct technology by 2024.

As we look at the current products, AMD’s hybrid bonding apparently focuses on stacking SRAM and computing units at the moment. However, as CPU leaders deepen their understanding of this technology, the application field is expected to further expand. In other words, the future of hybrid bonding solutions stacking multiple computation units is just around the corner.

Recently, there has been news of collaboration between NVIDIA and MediaTek. Speculation suggests that the future collaboration may extend to smartphone SoCs, allowing MediaTek to enhance the graphical computing and AI performance of Dimensity smartphone SoCs through NVIDIA’s GPU technology licensing.

Currently, the focus of this collaboration is primarily on NB SoC development, with some progress in the automotive-related chip sector. As for the scope of smartphone SoC collaboration, it is still under discussion, but the potential for related partnerships is worth noting.

In the announced collaboration between NVIDIA and MediaTek for the NB SoC products, MediaTek is mainly responsible for CPU, while other part such as GPU, DSP, ISP, and interface IP are provided by NVIDIA or external partners. NVIDIA holds the leadership position, while MediaTek plays a supporting role in this collaboration.

Regarding the industry’s speculation about possible collaboration in smartphone SoC development, it is estimated that MediaTek will take the lead in the design. Therefore, it is necessary to explore the motivations behind MediaTek’s adoption of related technologies.

Firstly, since the era of the Arm V9 instruction set, Arm’s reference GPU, Immortalis, has incorporated ray tracing functionality, assisting MediaTek’s flagship SoCs in improving gaming performance. This indicates that optimizing gaming scenarios is a key development focus for SoC manufacturers.

However, for high-end gaming applications, the current GPU performance of smartphone SoCs still cannot maintain high frame rates and native resolutions during gameplay. While selecting a pure core stacking approach to improve computational power is effective, it puts pressure on device power consumption. In light of this, Qualcomm introduced Snapdragon Game Super Resolution (GSR) technology this year, which simultaneously reduces power consumption and enhances game graphics quality. MediaTek has not yet explored this technology, and Arm Immortalis has not been released. Therefore, when it comes to GPU performance computing, MediaTek has incentives to seek external collaborations.

Furthermore, with the rapid upgrading of GPUs on smartphone SoCs, PC-level games are now being introduced to smartphones, and industry players are promoting compatibility with graphics APIs, opening doors for NVIDIA, AMD, and even Intel to enter the mobile gaming market. Samsung has partnered with AMD for its Exynos SoC GPU, while NVIDIA, with similar technology to Qualcomm Snapdragon GSR, becomes a logical choice as a cooperation partner for MediaTek.

TrendForce believes that if MediaTek integrates NVIDIA GPUs into Dimensity SoCs and leverages TSMC’s process power efficiency advantages, it could bring a new wave of excitement to MediaTek in the flagship or gaming device market, attracting consumer interest. However, despite the potential technical benefits of collaboration, considering the influence of geopolitical factors, MediaTek, which primarily sells its smartphone SoCs to Chinese customers, may ultimately abandon this collaboration option due to related policy risks.

According to the latest weekly memory spot prices published by TrendForce, the spot prices of DDR5 chips have returned to a downward trend, resulting in no signs of recovery in overall spot prices. For more details, please refer to the information below:

DRAM Spot Market:
Spot prices of DDR5 chips have swung down again, and spot prices of DDR4 and DDR3 products continue to register incremental declines on a daily basis. The spot market on the whole has yet to show signs of a price rebound. Lately, some buyers have been seeking quotes for small-quantity orders, but this kind of demand does not generate a sufficient momentum to expand the overall transaction volume. Presently, most traders generally believe that spot prices are almost at the bottom, but they remain passive in stocking up because the demand outlook is quite negative. The average spot price of the mainstream chips (i.e., DDR4 1Gx8 2666MT/s) fell by 0.97% from US$1.543 last week to US$1.528 this week.

NAND Flash Spot Market:
It is now a consensus within the market that prices are no longer able to drop further amidst the manifestation of efficacy from the diminished provision among suppliers. Spot prices are now starting to stabilize on the whole despite transactions having yet to magnify accordingly. 512Gb TLC wafer has dropped by 0.49% this week, arriving at US$1.420.

ChatGPT’s debut has sparked a thrilling spec upgrade in the server market, which has breathed new life into the supply chain and unlocked unparalleled business opportunities. Amidst all this, the big winners look set to be the suppliers of ABF (Ajinomoto Build-up Film) substrates, who are poised to reap enormous benefits.

In the previous article, “AI Sparks a Revolution Up In the Cloud,” we explored how the surge in data volumes is driving the spec of AI servers as well as the cost issue that comes with it. This time around, we’ll take a closer look at the crucial GPU and CPU platforms, focusing on how they can transform the ABF substrate market.

NVIDIA’s Dual-Track AI Server Chip Strategy Fuels ABF Consumption

In response to the vast data demands of fast-evolving AI servers, NVIDIA is leading the pack in defining the industry-standard specs.

This contrasts with standard GPU servers, where one CPU backs 2 to 6 GPUs. Instead, NVIDIA’s AI servers, geared towards DL(Deep Learning) and ML(Machine Learning), typically support 2 CPUs and 4 to 8 GPUs, thus doubling the ABF substrate usage compared to conventional GPU servers.

NVIDIA has devised a dual-track chip strategy, tailoring their offerings for international and Chinese markets. The primary chip for ChatGPT is NVIDIA’s A100. However, for China, in line with U.S. export regulations, they’ve introduced the A800 chip, reducing interconnect speeds from 600GBps (as on the A100) to 400GBps.

Their latest H100 GPU chip, manufactured at TSMC’s 4nm process, boasts an AI training performance 9 times greater than its A100 predecessor and inferencing power that’s 30 times higher. To match the new H100, H800 was also released with an interconnect speed capped at 300GBps. Notably, Baidu’s pioneering AI model, Wenxin, employs the A800 chip.

To stay competitive globally in AI, Chinese manufacturers are expected to aim for the computational prowess on par with the H100 and A100 by integrating more A800 and H800 chips. This move will boost the overall ABF substrate consumption.

With the ChatBot boom, it is predicted a 38.4% YoY increase in 2023’s AI server shipments and a robust CAGR of 22% from 2022 to 2026 – significantly outpacing the typical single-digit server growth, according to TrendForce’s prediction.

AMD, Intel Server Platforms Drive ABF Substrate Demand

Meanwhile, examining AMD and Intel’s high-end server platforms, we can observe how spec upgrades are propelling ABF substrate consumption forward.

• AMD Zen 4:

Since 2019, AMD’s EPYC Zen 2 server processors have used Chiplet multi-chip packaging, which due to its higher conductivity and cooling demands, has consistently bolstered ABF substrate demand.

• Intel Eagle Stream:

Intel’s advanced Eagle Stream Sapphire Rapids platform boasts 40-50% higher computation speed than its predecessor, the Whitley, and supports PCIe5, which triggers a 20% uptick in substrate layers. This platform employs Intel’s 2.5D EMIB tech and Silicon Bridge, integrating various chips to minimize signal transmission time.

The Sapphire Rapids lineup includes SPR XCC and the more advanced SPR HBM, with the latter’s ABF substrate area being 30% larger than the previous generation’s. The incorporation of EMIB’s Silicon Bridge within the ABF substrate increases lamination complexity and reduces overall yield. Simply put, for every 1% increase in Eagle Stream’s server market penetration, ABF substrate demand is projected to rise by 2%.

As the upgrades for server-grade ABF substrates continue to advance, production complexity, layer count, and area all increase correspondingly. This implies that the average yield rate might decrease from 60-70% to 40-50%. Therefore, the actual ABF substrate capacity required for future server CPU platforms will likely be more than double that of previous generations.

ABF Substrate Suppliers Riding the Tide

By our estimates, the global ABF substrate market size is set to grow from $9.3 billion in 2023 to $15 billion in 2026 – a CAGR of 17%, underscoring the tremendous growth and ongoing investment potential in the ABF supply chain.

Currently, Taiwanese and Japanese manufacturers cover about 80% of the global ABF substrate capacity. Major players like Japan’s Ibiden, Shinko and AT&S, along with Taiwan’s Unimicron, Nan Ya, and Kinsus all consider expanding their ABF substrate production capabilities as a long-term strategy.

As we analyzed in another piece, “Chiplet Design: A Real Game-Changer for Substrates,” despite the recent economic headwinds, capacity expansion of ABF substrate can still be seen as a solid trend, which is secured by the robust growth of high-end servers. Hence, the ability to precisely forecast capacity needs and simultaneously improve production yields will be the key to competitiveness for all substrate suppliers.

Read more:

• HBM Supply Leader SK Hynix’s Market Share to Exceed 50% in 2023 Due to Demand for AI Servers, Says TrendForce
• Shipments of AI Servers Will Climb at CAGR of 10.8% from 2022 to 2026, Says TrendForce

(Photo Credit: Google)

TrendForce’s latest research indicates that, as production cuts to DRAM and NAND Flash have not kept pace with weakening demand, the ASP of some products is expected to decline further in 2Q23. DRAM prices are projected to fall 13~18%; NAND Flash is expected to fall between 8~13%.

DRAM Spot Market

In the spot market, the trajectory of prices of DDR5 chips is going to be a highlight in the short term. Some sellers are now willing to lower quotes on DDR5 products that have been enjoying price hikes for many consecutive days. However, there have been few to no actual transactions, so TrendForce will continue to closely monitor changes in the prices of these products. As for spot prices of DDR3 and DDR4 products, they are still on a downward trajectory with no sign of easing. The average spot price of mainstream chips (i.e., DDR4 1Gx8 2666MT/s) fell by 1.33% from US$1.575 last week to US$1.554 this week.

NAND Flash Spot Market
Suppliers continue to enlarge in production cuts. Despite the insignificant increase in spot market demand, the continuously shrinking output of small-capacity wafers, as well as the attempt at price revitalization among a number of suppliers, have pulled up demand and transactions of low-priced inventory within the spot market, which led to a small price increment. However, mainstream wafer prices have yet to suspend in decline. 512Gb TLC wafer has dropped by 0.21% this week, arriving at US$1.427.