According to the news from Liberty Times Net, NVIDIA’s Q2 financials and Q3 forecasts have astounded the market, driven by substantial growth in their AI-centric data center operations. NVIDIA addresses CoWoS packaging supply issues by collaborating with other suppliers, boosting future capacity, and meeting demand. This move is echoed in South Korea’s pursuit of advanced packaging strategies.

South Korea’s Swift Pursuit on Advanced Packaging

The semiconductor industry highlights that the rapid development of generative AI has outpaced expectations, causing a shortage of advanced packaging production capacity. Faced with this supply-demand gap, TSMC has outsourced some of its capacity, with Silicon Interposer production being shared by facilities under the United Microelectronics Corporation and Siliconware Precision Industries. UMC has also strategically partnered with Siliconware Precision Industries, and Amkor’s Korean facilities have joined the ranks of suppliers to augment production capacity.

Due to equipment limitations, TSMC’s monthly CoWoS advanced packaging capacity is expected to increase from 10,000 units to a maximum of 12,000 units by the end of this year. Meanwhile, other suppliers could potentially raise their CoWoS monthly capacity to 3,000 units. TSMC aims to boost its capacity to 25,000 units by the end of next year, while other suppliers might elevate theirs to 5,000 units.

According to the source South Korean media, Samsung entered the scene, competing for advanced packaging orders against NVIDIA. South Korea initiated a strategic research project to rapidly narrow the gap in packaging technology within 5~7 years, targeting giants like TSMC, Amkor, and China’s JCET.

(Source: https://ec.ltn.com.tw/article/paper/1601162)

According to a report from Taiwan’s Commercial Times, NVIDIA is aggressively establishing a non-TSMC CoWoS supply chain. Sources in the supply chain reveal that UMC is proactively expanding silicon interposer capacity, doubling it in advance, and now planning to further increase production by over two times. The monthly capacity for silicon interposers will surge from the current 3 kwpm (thousand wafers per month) to 10 kwpm, potentially aligning its capacity with TSMC’s next year, significantly alleviating the supply strain in the CoWoS process.

A prior report from Nomura Securities highlighted NVIDIA’s efforts since the end of Q2 this year to construct a non-TSMC supply chain. Key players include UMC for wafer fabrication, Amkor and SPIL for packaging and testing. NVIDIA aims to add suppliers to meet the surging demand for CoWoS solutions.

The pivotal challenge in expanding CoWoS production lies in insufficient silicon interposer supply. In the future, UMC will provide the silicon interposers for front-end CoW process, while Amkor and SPLI will take charge of the back-end WoS packaging. These collaborations will establish a non-TSMC CoWoS supply chain.

UMC states its current silicon interposer capacity stands at 3 kwpm. However, the company has decided to undertake a one-fold expansion at its Singaporean plant, targeting a capacity of around 6 kwpm. The additional capacity is anticipated to be progressively operational within 6 to 9 months, with the earliest projections for the first quarter of next year.

Yet, due to persistent robust market demand, it’s expected that even with UMC’s capacity expansion to 6 kwpm, it may not completely meet market needs. Consequently, industry sources suggest UMC has opted to further amplify silicon interposer capacity to 10 kwpm, aiming for a two-fold acceleration of production expansion. Addressing these expansion rumors, UMC affirms that growth in advanced packaging demand is an inherent trend and future focus, asserting their evaluation of capacity options and not ruling out the possibility of continuous enlargement of silicon interposer capabilities.

(Photo credit: Amkor)

According to a report by Taiwan’s Economic Daily, the latest GH200 module released by NVIDIA has seen its assembly orders exclusively undertaken by Foxconn, while the assembly orders for L40S are also entirely managed by Foxconn.

Foxconn has traditionally refrained from commenting on individual business and order dynamics. It is believed that AI chip modules constitute the highest-margin product within the entire server supply chain.

Foxconn has been a longstanding partner of NVIDIA, providing an end-to-end solution across chip modules, baseboards, motherboards, servers, and chassis. Foxconn’s capabilities have facilitated the creation of a comprehensive solution for NVIDIA’s AI server supply chain.

Previously, Foxconn had an exclusive assembly partnership with NVIDIA for the “H100” and “H800” modules, not only retaining the existing orders but also securing a substantial portion of the HGX module orders. Now, reports indicate that Foxconn will exclusively supply even NVIDIA’s newly unveiled GH 200, and the L40S.

Industry sources indicate that due to severe constraints on TSMC’s advanced CoWoS packaging capacity, the scaling up of NVIDIA’s AI chip production has been hindered. However, with new CoWoS production capacity set to gradually open up in the late third quarter to the fourth quarter, shipments of Foxconn’s AI chip modules are anticipated to rapidly increase.

Industry sources reveal that in business negotiations, NVIDIA is known for demanding from its suppliers, but it is also generous in its offerings. As long as suppliers provide products that meet or even exceed expectations, NVIDIA is willing to offer reasonable prices, fostering mutually beneficial relationships with its partners.

(Photo credit: NVIDIA)

According to the Korea Economic Daily. Samsung Electronics’ HBM3 and packaging services have passed AMD’s quality tests. The upcoming Instinct MI300 series AI chips from AMD are planned to incorporate Samsung’s HBM3 and packaging services. These chips, which combine central processing units (CPUs), graphics processing units (GPUs), and HBM3, are expected to be released in the fourth quarter of this year.

Samsung is noted as the sole provider capable of offering advanced packaging solutions and HBM products simultaneously. Originally considering TSMC’s advanced packaging services, AMD had to alter its plans due to capacity constraints.

The surge in demand for high-performance GPUs within the AI landscape benefits not only GPU manufacturers like NVIDIA and AMD, but also propels the development of HBM and advanced packaging.

In the backdrop of the AI trend, AIGC model training and inference require the deployment of AI servers. These servers typically require mid-to-high-end GPUs, with HBM penetration nearing 100% among these GPUs.

Presently, Samsung, SK Hynix, and Micron are the primary HBM manufacturers. According to the latest research by TrendForce, driven by the expansion efforts of these original manufacturers, the estimated annual growth rate of HBM supply in 2024 is projected to reach 105%.

In terms of competitive dynamics, SK Hynix leads with its HBM3 products, serving as the primary supplier for NVIDIA’s Server GPUs. Samsung, on the other hand, focuses on fulfilling orders from other cloud service providers. With added orders from customers, the gap in market share between Samsung and SK Hynix is expected to narrow significantly this year. The estimated HBM market share for both companies is about 95% for 2023 to 2024. However, variations in customer composition might lead to sequential variations in bit shipments.

In the realm of advanced packaging capacity, TSMC’s CoWoS packaging technology dominates as the main choice for AI server chip suppliers. Amidst strong demand for high-end AI chips and HBM, TrendForce estimates that TSMC’s CoWoS monthly capacity could reach 12K by the end of 2023.

With strong demand driven by NVIDIA’s A100 and H100 AI Server requirements, demand for CoWoS capacity is expected to rise by nearly 50% compared to the beginning of the year. Coupled with the growth in high-end AI chip demand from companies like AMD and Google, the latter half of the year could experience tighter CoWoS capacity. This robust demand is expected to continue into 2024, potentially leading to a 30-40% increase in advanced packaging capacity, contingent on equipment readiness.

(Photo credit: Samsung)

US-based CSPs have been establishing SMT production lines in Southeast Asia since late 2022 to mitigate geopolitical risks and supply chain disruptions. TrendForce reports that Taiwan-based server ODMs, including Quanta, Foxconn, Wistron (including Wiwynn), and Inventec, have set up production bases in countries like Thailand, Vietnam, and Malaysia. It’s projected that by 2023, the production capacity from these regions will account for 23%, and by 2026, it will approach 50%.

TrendForce reveals that Quanta, due to its geographical ties, has established several production lines in its Thai facilities centered around Google and Celestica, aiming for optimal positioning to foster customer loyalty. Meanwhile, Foxconn has renovated its existing facilities in Hanoi, Vietnam, and uses its Wisconsin plant to accommodate customer needs. Both Wistron and Wiwynn are progressively establishing assembly plants and SMT production lines in Malaysia. Inventec’s current strategy mirrors that of Quanta, with plans to build SMT production lines in Thailand by 2024 and commence server production in late 2024.

CSPs aim to control the core supply chain, AI server supply chain trends toward decentralization

TrendForce suggests that changes in the supply chain aren’t just about circumventing geopolitical risks—equally vital is increased control over key high-cost components, including CPUs, GPUs, and other critical materials. With rising demand for next-generation AI and Large Language Models, supply chain stockpiling grows each quarter. Accompanied by a surge in demand in 1H23, CSPs will become especially cautious in their supply chain management.

Google, with its in-house developed TPU machines, possesses both the core R&D and supply chain leadership. Moreover, its production stronghold primarily revolves around its own manufacturing sites in Thailand. However, Google still relies on cooperative ODMs for human resource allocation and production scheduling, while managing other materials internally. To avoid disruptions in the supply chain, companies like Microsoft, Meta, and AWS are not only aiming for flexibility in supply chain management but are also integrating system integrators into ODM production. This approach allows for more dispersed and meticulous coordination and execution of projects.

Initially, Meta heavily relied on direct purchases of complete server systems, with Intel’s Habana system being one of the first to be integrated into Meta’s infrastructure. This made sense since the CPU for their web-type servers were often semi-custom versions from Intel. Based on system optimization levels, Meta found Habana to be the most direct and seamless solution. Notably, it was only last year that Meta began to delegate parts of its Metaverse project to ODMs. This year, as part of its push into generative AI, Meta has also started adopting NVIDIA’s solutions extensively.