Charlie Boyle, Vice President of NVIDIA’s DGX Systems, recently addressed the issue of limited GPU production at the company.

Boyle clarified that the current GPU shortage is not a result of NVIDIA misjudging demand or constraints in Taiwan Semiconductor Manufacturing Company’s (TSMC) wafer production. The primary bottleneck for GPUs lies in the packaging process.

It’s worth noting that the NVIDIA A100 and H100 GPUs are currently manufactured by TSMC using their advanced CoWoS (Chip-on-Wafer-on-Substrate) packaging technology. TSMC has indicated that it may take up to a year and a half, including the completion of additional wafer fabs and expansion of existing facilities, to normalize the backlog of packaging orders.

Furthermore, due to the significant strain on TSMC’s CoWoS capacity, there have been reports of overflow of NVIDIA GPU packaging orders to other manufacturers.

Sources familiar with the matter have revealed that NVIDIA is in discussions with potential alternative suppliers, including Samsung, as secondary suppliers for the 2.5D packaging of NVIDIA’s A100 and H100 GPUs. Other potential suppliers include Amkor and the Siliconware Precision Industries Co., Ltd. (SPIL), a subsidiary of ASE Technology Holding.

In December 2022, Samsung established its Advanced Packaging (AVP) division to seize opportunities in high-end packaging and testing. Sources suggest that if NVIDIA approves of Samsung’s 2.5D packaging process yield, a portion of AI GPU packaging orders may be placed with Samsung.

TrendForce’s research in June this year indicated that driven by strong demand for high-end AI chips and High-Bandwidth Memory (HBM), TSMC’s CoWoS monthly capacity could reach 12,000 units by the end of 2023. Particularly, demand from NVIDIA for A100 and H100 GPUs in AI servers has led to nearly a 50% increase in CoWoS capacity compared to the beginning of the year. Coupled with the growth in demand for high-end AI chips from companies like AMD and Google, the second half of the year is expected to witness tighter CoWoS capacity. This robust demand is projected to continue into 2024, with advanced packaging capacity potentially growing by 30-40% if the necessary equipment is in place.

(Photo credit: NVIDIA)

According to the latest report from TrendForce, the primary factors influencing the global market share of notebook CPUs in 2024 can be categorized into “Architectural Design” and “Economic Factors.”

“Architectural Design” as a long-term factor affecting market share:

(1) Both AMD (AMD 3D V-Cache) and Intel (Intel Foveros Direct) may potentially integrate 3D packaging technology into notebook computers in the future.

(2) Apple’s M-series processors, using the Arm core architecture, as well as Intel processors, have adopted a big/little core hybrid design. AMD might also introduce this in the Ryzen 8000 series.

(3) Despite further advancements in processor technology by 2024, the notebook computer market remains highly sensitive to the cost for IT equipment.

“Economic Factors” as more immediate influencers of market share:

(1) Until 2024, a return to lower interest rates in the global economic environment could favor corporate expansion of capital expenditure. This could result in increased procurement of business-oriented notebook models, potentially allowing Intel to further expand its CPU market share beyond 70% in the business sector.

(2) Concerns about economic prospects among global citizens until 2024 could have significant negative implications for the consumer notebook computer market. With the restart of physical economic activities, the demand for consumer-oriented notebook models has declined from the high levels seen during the pandemic. Consequently, the consumer market demand outlook for 2024 remains uncertain. For AMD, which relies more on consumer market demand, changes in market share may be harder to predict compared to Intel.

In the post-pandemic era, AMD, Arm/Apple, and Intel are pursuing distinct technological competition strategies to capture market share in the personal computing market.

AMD:

(1) The Socket AM5 platform is poised to aid AMD CPUs in achieving substantial performance and efficiency gains.

(2) The AMD Ryzen 7040 incorporates an artificial intelligence engine to emphasize AI computing performance’s importance in the thin and light notebook market.

Arm/Apple:

(1) The M2 Ultra processor heralds Apple’s complete transition of personal computing products to the Arm core. Apple Mac computer products will no longer be sold with Intel processor.

(2) The Apple M-series processors, built on the Arm core architecture, facilitate a “fanless design” to maintain MacBook’s slim profile. This feature highlights its irreplaceable positioning in the portable notebook computer market, emphasizing portability.

Intel:

(1) With the waning trend of the “hybrid work mode,” Intel is optimistic about diversified development in the post-pandemic era for desktop computer products. This includes microcomputers, micro workstations, and general workstations. Due to the characteristic of continuous operation for 24 hours, desktop computers still possess unique attributes that cannot be replaced by notebook computers.

(Photo credit: Intel)

The surge in AIGC and new technologies such IoT, AI, 5G, AR/VR are driving a huge demand for computational power of high-end chips. This has been even outpacing the performance increase offered by the long-standing Moore’s Law, ushering in a “post-Moore” era where revolutions in advanced chip design are crucial.

Over recent years, chiplet design has seemingly become the mainstream approach for upgrading high-end chips. The concept is to allow more transistors on a single chip, effectively increasing the production yield of high-end chips while reducing overall costs.

By the large, major IC players have all jumped on board. Even Apple has joined the game by releasing their M1 Ultra SoC using the chiplet concept, which doubles computational performance by integrating two M1 Max units in a single chip.

The CPU sector is definitely a clear demonstration of this trend:

• AMD took the leap with chiplet design in their 2nd-gen EPYC CPUs, doubling the computing cores from 32 to 64 within two years, while slashing costs by up to half. The company has extended this approach to their 4th-gen EPYC CPUs and even pioneered the GPU Navi 31, the first of its kind to use chiplets.
• Intel started incorporating chiplets into their Lakefield series SoC in 2020. Looking ahead, their upcoming CPUs like the Meteor Lake set for 2023, and Arrow Lake and Lunar Lake scheduled for 2024, will all use chiplet design.

Transition from Bumping to Hybrid Bonding

Our analysis in “Chiplet Design: A Real Game-Changer for Substrates” laid out the comprehensive impact of the evolution of chiplet technology on substrates. In fact, chiplets have already caused a significant disruption to the most advanced semiconductor packaging technologies, necessitating the transition towards advanced 2.5D and 3D packaging technologies.

The bottleneck of advanced packaging lies in the chiplets’ interconnections, with bump and microbump still being the key technology for linking chips and forming I/O joints. These connection densities are hard to enhance, thus limiting the overall chip’s transmission speed. In addition, the more chiplets being stacked, the bigger the chip volume gets. The challenge is how to limit the chip size within a specific range, considering the current technical constraints.

Therefore, copper-to-copper hybrid bonding, also known as DBI (Direct Bond Interconnect), has been emerging as the key technology route that overcomes major hurdles in chiplet integration from the bottom-up.

Unlike bumping technology, hybrid bonding significantly shrinks the I/O joint space. The future transmission demand requires the I/O joint space between chiplets to be less than 10µm. While bumping is limited to around 20µm, hybrid bonding can take this down to an impressive 1µm or even less. This also means more I/O joints can be fitted in the same chip size – even reaching up to millions on a mere 1cm2 chip.

On top of this, hybrid bonding only adds an extra 1-2µm of thickness, compared to the 10-30µm of microbump, thereby helping reduce the thickness of stacked chips.

To put it simply, hybrid bonding can boost transmission efficiency, minimize energy usage with higher density of copper joints, manage chip volume, and even cut down on material costs.

The Race for Advanced Packaging Is Kicking Off

Moving forward, hybrid bonding is set to become the key technology supporting the continuous development of chiplet design and 3D packaging. This has been exemplified by TSMC’s front-end So IC packaging technology which is based on hybrid bonding. This puts AMD, a key customer of TSMC, in a favorable position to get ahead.

From AMD’s roadmap of 3D V-Cache technology, they have stacked SRAM on top of CCX (CPU Complex), and gradually integrated it into Milan-X series, the EPYC server CPUs, and Ryzen series, the consumer-grade CPUs, over the past two years. This has significantly improved performance and power consumption as a whole.

Not to be outdone, this year Intel also launched their Foveros Direct packaging technology, which is also based on hybrid bonding route. Assuming everything proceeds smoothly, we can anticipate the release of CPUs utilizing Foveros Direct technology by 2024.

As we look at the current products, AMD’s hybrid bonding apparently focuses on stacking SRAM and computing units at the moment. However, as CPU leaders deepen their understanding of this technology, the application field is expected to further expand. In other words, the future of hybrid bonding solutions stacking multiple computation units is just around the corner.

ChatGPT’s debut has sparked a thrilling spec upgrade in the server market, which has breathed new life into the supply chain and unlocked unparalleled business opportunities. Amidst all this, the big winners look set to be the suppliers of ABF (Ajinomoto Build-up Film) substrates, who are poised to reap enormous benefits.

In the previous article, “AI Sparks a Revolution Up In the Cloud,” we explored how the surge in data volumes is driving the spec of AI servers as well as the cost issue that comes with it. This time around, we’ll take a closer look at the crucial GPU and CPU platforms, focusing on how they can transform the ABF substrate market.

NVIDIA’s Dual-Track AI Server Chip Strategy Fuels ABF Consumption

In response to the vast data demands of fast-evolving AI servers, NVIDIA is leading the pack in defining the industry-standard specs.

This contrasts with standard GPU servers, where one CPU backs 2 to 6 GPUs. Instead, NVIDIA’s AI servers, geared towards DL(Deep Learning) and ML(Machine Learning), typically support 2 CPUs and 4 to 8 GPUs, thus doubling the ABF substrate usage compared to conventional GPU servers.

NVIDIA has devised a dual-track chip strategy, tailoring their offerings for international and Chinese markets. The primary chip for ChatGPT is NVIDIA’s A100. However, for China, in line with U.S. export regulations, they’ve introduced the A800 chip, reducing interconnect speeds from 600GBps (as on the A100) to 400GBps.

Their latest H100 GPU chip, manufactured at TSMC’s 4nm process, boasts an AI training performance 9 times greater than its A100 predecessor and inferencing power that’s 30 times higher. To match the new H100, H800 was also released with an interconnect speed capped at 300GBps. Notably, Baidu’s pioneering AI model, Wenxin, employs the A800 chip.

To stay competitive globally in AI, Chinese manufacturers are expected to aim for the computational prowess on par with the H100 and A100 by integrating more A800 and H800 chips. This move will boost the overall ABF substrate consumption.

With the ChatBot boom, it is predicted a 38.4% YoY increase in 2023’s AI server shipments and a robust CAGR of 22% from 2022 to 2026 – significantly outpacing the typical single-digit server growth, according to TrendForce’s prediction.

AMD, Intel Server Platforms Drive ABF Substrate Demand

Meanwhile, examining AMD and Intel’s high-end server platforms, we can observe how spec upgrades are propelling ABF substrate consumption forward.

• AMD Zen 4:

Since 2019, AMD’s EPYC Zen 2 server processors have used Chiplet multi-chip packaging, which due to its higher conductivity and cooling demands, has consistently bolstered ABF substrate demand.

• Intel Eagle Stream:

Intel’s advanced Eagle Stream Sapphire Rapids platform boasts 40-50% higher computation speed than its predecessor, the Whitley, and supports PCIe5, which triggers a 20% uptick in substrate layers. This platform employs Intel’s 2.5D EMIB tech and Silicon Bridge, integrating various chips to minimize signal transmission time.

The Sapphire Rapids lineup includes SPR XCC and the more advanced SPR HBM, with the latter’s ABF substrate area being 30% larger than the previous generation’s. The incorporation of EMIB’s Silicon Bridge within the ABF substrate increases lamination complexity and reduces overall yield. Simply put, for every 1% increase in Eagle Stream’s server market penetration, ABF substrate demand is projected to rise by 2%.

As the upgrades for server-grade ABF substrates continue to advance, production complexity, layer count, and area all increase correspondingly. This implies that the average yield rate might decrease from 60-70% to 40-50%. Therefore, the actual ABF substrate capacity required for future server CPU platforms will likely be more than double that of previous generations.

ABF Substrate Suppliers Riding the Tide

By our estimates, the global ABF substrate market size is set to grow from $9.3 billion in 2023 to $15 billion in 2026 – a CAGR of 17%, underscoring the tremendous growth and ongoing investment potential in the ABF supply chain.

Currently, Taiwanese and Japanese manufacturers cover about 80% of the global ABF substrate capacity. Major players like Japan’s Ibiden, Shinko and AT&S, along with Taiwan’s Unimicron, Nan Ya, and Kinsus all consider expanding their ABF substrate production capabilities as a long-term strategy.

As we analyzed in another piece, “Chiplet Design: A Real Game-Changer for Substrates,” despite the recent economic headwinds, capacity expansion of ABF substrate can still be seen as a solid trend, which is secured by the robust growth of high-end servers. Hence, the ability to precisely forecast capacity needs and simultaneously improve production yields will be the key to competitiveness for all substrate suppliers.

Read more:

• HBM Supply Leader SK Hynix’s Market Share to Exceed 50% in 2023 Due to Demand for AI Servers, Says TrendForce
• Shipments of AI Servers Will Climb at CAGR of 10.8% from 2022 to 2026, Says TrendForce

(Photo Credit: Google)

In the post-Moore’s Law era, chiplet design has been burgeoning as the mainstream architecture.

With the widespread adoption of EUV technology by foundries on process nodes of 5nm and below, the cost of semiconductor fabrication has skyrocketed. The cost of the 5nm process has grown by almost 1x compared to the 7nm process, and the 3nm process is expected to increase by almost 1x compared to the 5nm process.

To address this issue, IC design companies have started to split chip components or connect multiple chips and adopt advanced packaging such as 2.5D/3D IC to integrate multiple chips together.

Compared to traditional chip design methods, chiplet design has superior characteristics such as shorter upgrade cycles, lower costs, and higher yields, which is one of the reasons why chiplet technology is gaining popularity.

AMD’s chiplet design is a representative example. Through close collaboration with TSMC, AMD has fully transitioned its CPUs to chiplets since the 7nm process, with the Ryzen 7000 series CPU and Radeon RX 7000 series graphics cards released in 2022. The latter uses the RDNA 3 architecture and integrates the GCD and MCD produced by the 5nm and 6nm processes respectively, as a result improving overall performance, with a 54% increase in RDNA 3’s Performance per Watt.

Under the leadership of industry leaders such as AMD and Intel, chiplet design has had a significant impact on the entire semiconductor industry – substrates manufacturers in particular.

ABF Substrates Set to Soar

Aside from CPUs, developments in AMD and Intel’s server platforms indicate that the trend towards higher-layer-count and larger-area ABF substrates is expected to continue.

Given the server shipment volume is expected to remain stable and grow steadily in the mid to low single digits for the next 3-5 years, the growth momentum of ABF substrates mainly comes from the increase in layer count and area brought by 2.5D/3D packaging adoption in servers.

Starting in 2020, ABF substrates saw a surge in demand due to the pandemic. The supply-demand gap peaked in 2021, and in the first half of 2022, ABF substrate prices increased while volume increased and gross profit margins hit new highs.

Due to the impact of shortage in ABF substrates in 2020-2021, major substrate manufacturers have initiated large-scale expansion plans, with the expectation that demand for ABF substrates would continue to grow with the upcoming releases of new server platforms and the integration of 2.5D packaging for PC CPUs.

Growing demands with Some Hiccups

However, the moves have been put on hold for now. Since the second half of 2022, due to inventory correction in the overall semiconductor industry and the delayed production time of Intel’s new server platform, there’s been a supply glut in ABF substrates.

Therefore, Unimicron has taken the lead in adjusting its capital expenditure plans, reducing its planned capacity increase for 2023 from about 20% to only 3.5%. AT&S has also tentatively postponed the significant increase in capacity planned for the end of 2024. It is unclear when the expansion will resume or whether the expansion will be scaled back.

This indicates that current substrate manufacturers have not only lowered their demand projections for 2023, but also for 2025-2026. Further adjustments to the expansion plans of other manufacturers will also affect the future market supply-demands dynamics.

Back on Track for Major Growth in 2024

Looking into the future, things are looking up for the ABF substrate industry. In the second quarter of 2023, we can expect the release of new server platforms from AMD and Intel, as well as the completion of PC inventory adjustments.

With expansion plans in place, it’s predicted that global ABF substrate production capacity will only increase by 15-20% in the latter half of 2023, continuing to put pressure on substrate manufacturers, according to TrendForce.

Things are expected to pick up in 2024 with the release of AMD and Intel’s next-generation server platforms, Zen 5 and Birch Stream. Plus, the anticipated introduction of 2.5D packaging for PC CPUs will drive a new wave of demand for ABF substrates. All in all, we can expect a significant rebound for the ABF substrate industry in 2024.