According to a news report from IJIWEI, sources have revealed that NVIDIA has placed urgent orders with TSMC for the production of AI GPU destined for China. These orders fall under the category of “Super Hot Run” (SHR), with plans to commence fulfillment in the first quarter of 2024.

Respond to the United States implementing stricter export controls on the Chinese semiconductor industry, sources stated in the report indicate that NVIDIA plans to provide a new “specialized” AI chip to China by lowering specifications, replacing the export-restricted H800, A800, and L40S series.

Insiders suggest that NVIDIA intends to resume supplying the RTX 4090 chip to China in January of next year but also release a modified version later to comply with U.S. export restrictions. 

On the other hand, NVIDIA continues to increase its orders with TSMC. This move aims to secure TSMC’s manufacturing capacity to meet the demand for the H100. However, due to limitations in CoWoS (Chip-on-Wafer-on-Substrate) production capacity, the H100 GPU is currently facing severe shortages.

It is noted that following NVIDIA, Intel and AMD are also expected to tailor AI chips for China. TSMC, as the primary pure-play foundry partner for these AI chip suppliers, will continue to enjoy a competitive advantage.

According to sources from semiconductor equipment manufacturers, despite TSMC’s efforts to increase CoWoS production capacity, the foundry still cannot meet the growing demand for NVIDIA GPUs. Additionally, the MI300 chip that was recently launched by AMD is also competing for the foundry industry’s production capacity.

Insiders note that TSMC’s ability to expand CoWoS production capacity is limited, with delays in equipment replacement speed, machine installation speed, and labor deployment. The new capacity is expected to be ready by the second quarter of 2024.

Equipment is identified as one of the key variables affecting TSMC’s expansion of CoWoS production capacity. Unexpected impacts on production and delivery times from Japanese equipment supplier Shibaura have delayed the development and installation of new capacity across TSMC’s production lines, including those in Longtan and Zhunan.

TSMC Chairman Mark Liu mentioned in a press conference in September that the shortage of CoWoS packaging capacity at TSMC is temporary, and it will be addressed through capacity expansion within the next year and a half to meet the growing demand.

(Photo credit: TSMC)

Read more

• [News] TSMC’s CoWoS Demand Surges from NVIDIA, Apple, AMD, Broadcom, Marvell, Monthly Capacity Up 120% in 2024

Nvidia CEO Jensen Huang announced on the 11th the company’s intention to deepen collaboration with high-tech companies in Vietnam, with a focus on fostering local expertise in AI and digital infrastructure development. Huang revealed plans to establish a chip center in Vietnam, as reported by Reuters.

According to documents released by the White House in September to enhance bilateral relations, Nvidia has invested USD 250 million in Vietnam. The company has strategically aligned with leading tech companies to implement AI technology in cloud computing, automotive, and healthcare industries.

This marks Huang’s first visit to Vietnam, where, during an event in Hanoi, he emphasized, “Vietnam is already our partner as we have millions of clients here.” He stated, “Vietnam and Nvidia will deepen our relations, with Viettel, FPT, Vingroup, VNG being the partners Nvidia looks to expand partnership with,” Huang said, adding Nvidia would support Vietnam’s artificial training and infrastructure.

Vietnam’s Minister of Planning and Investment, Chi Dung Nguyen, highlighted during the meeting on December 11th the country’s ongoing efforts to design mechanisms and incentives to attract investments in semiconductor and AI projects.

During his meeting with Vietnamese Prime Minister Pham Minh Chinh on the 10th, Huang shared the vision of establishing an R&D center, emphasizing that “the base will be for attracting talent from around the world to contribute to the development of Vietnam’s semiconductor ecosystem and digitalization.” Subsequently, on the 11th, Nguyen Chi Dung extended an invitation for Nvidia to consider establishing an R&D base in the country.

On the 11th, Nvidia engaged in discussions with the Vietnamese government and local tech companies regarding semiconductor cooperation agreements. According to insiders, Nvidia may potentially reach a technology transfer agreement with at least one Vietnamese company.

Given the strained trade relations between China and the U.S., Vietnam’s technology and manufacturing sectors are presented with a significant opportunity. The government actively seeks to enhance chip design capabilities and explore avenues for establishing a viable chip manufacturing industry.

Vietnam already serves as a pivotal IC packaging hub for global chip manufacturers. For instance, Intel boasts that it has world’s largest IC packaging and testing facility, is situated in Vietnam. Despite temporary delays in the expansion of its Vietnamese factory due to power supply and bureaucratic challenges, Intel affirmed in a Reuters interview, “Vietnam will continue to be a critical part of our global manufacturing operations as demand for semiconductors grows.”

Furthermore, several chipmakers have recently set up or expanded production facilities in Vietnam. Major OSAT provider Amkor commenced operations at its new USD 1.6 billion IC packaging plant in Yen Phong 2C Industrial Park, Bac Ninh Province, Vietnam, in October this year. A month earlier, Samsung’s OSAT partner, Hana Micron, announced the inauguration of its USD 600 million IC packaging plant in Bac Giang Province.

(Image: Nvidia)

Explore more

• [News] Amkor’s Vietnam Plant Meets High-End Semiconductor Demand as Advanced Packaging Thrives

Nvidia CFO, Colette Kress, recently hinted again that the next-gen chips might be outsourced to Intel Corp. During the call with semiconductor analyst Tim Arcuri at the UBS Global Technology Conference on November 28th, she was asked whether Intel would be considered as a foundry partner for the next-gen chips.

In response, she stated that there are many powerful foundries in the market. TSMC and Samsung Electronics have been great partners. She said, “we’d love to have a third one,” when answering whether Nvidia want a third partner.

Kress also mentioned that, TSMC’s and others’ US fab may also be their options, and “there is nothing necessarily different but again in terms of different region. Nothing will stop us from potentially adding another foundry.”

Kress highlighted that Nvidia’s current data center GPUs designed for AI and high-performance computing (HPC) are predominantly outsourced to TSMC. However, in the previous generation, Nvidia’s gaming GPUs were mainly entrusted to Samsung for fabrication. According to Sedaily, Samsung’s foundry was responsible for manufacturing Nvidia’s GeForce RTX 30 series gaming GPUs based on the Ampere architecture.

Speaking of foundry partners for AI products, Nvidia anticipates that TSMC will remain a crucial foundry partner for producing AI Hopper H200 and Blackwell B100 GPUs. Any additional orders might be entrusted to Samsung.

Nvidia CEO previously said Intel’s next-gen process test chips “look good”

Additionally, reports from Barron also mentioned that on May 30th, during an interaction with journalists in Taiwan, Nvidia CEO Jensen Huang was asked whether Nvidia is considering diversifying its supplier base given the rising tensions between the U.S. and China. In response, Huang referred to Nvidia’s long-standing collaboration with TSMC and Samsung Electronics, stating, “We have a lot of customers depending on us. And so our supply chain resilience is very important to us. We manufacture in as many places as we can.”

At that time, Huang also expressed, “We’re open to manufacturing with Intel. And (Intel CEO) Pat (Gelsinger) has said in the past that we’re evaluating their process, and we’ve recently received the test chip results of their next generation process and the results look good.”

From Nvidia CFO’s talk in November and Nvidia CEO’s response in May, it is obvious that, beyond TSMC and Samsung, Nvidia is thinking about a potential third foundry partner.

(Image: NVIDIA Hopper Architecture – H100 SXM)

On November 22, 2023, NVIDIA released its financial report for the third quarter of 2023 (FY3Q24: August 2023 to October 2023), with a revenue of USD 18.1 billion. This represents a quarterly increase of 34% and a yearly increase of 206%.

NVIDIA also provided a revenue guidance for the fourth quarter of 2023 (FY4Q24: November 2023 to January 2024), with a median estimate of USD 20 billion. This reflects a quarterly increase of 10.5% and a yearly increase of 231%.

TrendForce’s Insight:

1. Downgraded Chips Expected by Year-End 

NVIDIA continues its robust performance in the third quarter of 2023 (FY3Q24). The Datacenter division reported a revenue of USD 14.5 billion, marking a 41% quarterly increase and a staggering 279% annual increase. This segment now constitutes 80% of the overall revenue, with a 4% increase from the previous quarter.

The growth is primarily driven by the HGX Hopper GPU, along with the commencement of shipments and revenue recognition for L40S and GH200. Approximately 50% of Datacenter revenue comes from CSP customers, while the remaining 50% is contributed by Consumer Internet and enterprise clients.

In terms of revenue outlook, the China region accounts for approximately 20-25% of NVIDIA’s Datacenter revenue. The management acknowledges the significant impact of the U.S. restrictions on China’s revenue for the fourth quarter of 2023 but expresses confidence that revenue from other regions can offset this impact.

This confidence stems from the current high demand and low supply situation for AI chips. Notably, NVIDIA’s anticipated release of lower-capacity chips, including HGX H20, L20 PCIe, and L2PCIe, originally slated for November 16, 2023, is now expected to be delayed until the end of 2023, presumably due to ongoing negotiations with the U.S. Department of Commerce.

2. NVIDIA Unveils HGX H200, High-Capacity Version of H100; Next-Gen B100 Anticipates Doubling Performance

In a significant product announcement, NVIDIA introduced HGX H200 on November 14, 2023. The new offering is a high-capacity version of H100 and is fully compatible with HGX H100 systems. This compatibility ensures that manufacturers using H100 don’t need to modify server systems or software specifications when transitioning to H200.

HGX H200 is slated for release in the second quarter of 2024, with initial customers including AWS, Microsoft Azure, Google Cloud, and Oracle Cloud. Additionally, CoreWeave, Lambda, and Vultr are expected to adopt HGX H200 in their setups.

Comparing H200 SXM with H100 SXM, it’s evident that H200 SXM has a 76% increase in memory capacity and a 43% increase in bandwidth compared to H100 SXM. Additionally, it upgrades from HBM3 to HBM3e, while the remaining specifications remain the same. This indicates that H200 SXM is essentially a high-capacity version of H100 SXM.

Given the sensitivity of performance to memory capacity and bandwidth, inference on the Llama2 with 7 billion parameters shows that the performance of H200 SXM can reach 1.9 times that of H100 SXM.

Moreover, NVIDIA plans to launch the B100, truly representing the next generation of products with the new Blackwell architecture in 2024. Utilizing Chiplet technology, it is speculated that the architecture will transition from a single die + 6 HBM3 configuration in H100 to a dual-die + 8 HBM3e configuration in B100, potentially doubling the performance compared to H100.

Read more

• [News] H200 Unveiled: NVIDIA Integrates HBM3e for Enhanced AI Performance

(Photo credit: NVIDIA)

The AI landscape witnesses a robust surge with the consecutive launches of AMD’s “Instinct MI300” series AI chips and NVIDIA’s upcoming “B100” GPU structure. This wave of innovation propels a flourishing demand for AI-related Outsourced Semiconductor Assembly And Test Services (OSAT), surpassing initial estimates by over 10%. OSAT companies like ASE Holdings, King Yuan Electronics (KYEC), and Sigurd are poised to experience a notable uptick in revenue, as reported by UDN News.

According to reports, AMD is launching the “Instinct MI300” series AI chips this week, and NVIDIA plans to unveil the next-gen “B100” GPU next year. This successive release of new AI products by the two giants is boosting momentum in related OSATs collaboration.

NVIDIA is gearing up for the 2024 launch of its next-gen Blackwell architecture B100 GPU, saying AI performance exceeding twice that of the H200 GPU under the Hopper architecture, signifying a substantial leap in computational prowess.

Positive Outlook in 2024 for OSATs Amid AI Chip Development

Industry source indicates that due to the AI extensive computation requirements, advanced packaging is gradually becoming mainstream. This involves stacking chips and packaging them on a substrate. Depending on the arrangement, it is divided into 2.5D and 3D packaging. The advantage of this packaging technology is the ability to reduce chip space while also reducing power consumption and costs.

It is said the surge in AI chip orders from AMD and NVIDIA has led to a bottleneck in TSMC CoWoS advanced packaging capacity. This unexpected demand has exceeded projections for related OSATs, including ASE Holdings, KYEC, and Sigurd.

In the case of ASE Holdings, its subsidiary Siliconware Precision Industries (SPIL) possesses the advanced packaging capacity essential for generative AI chips. Joseph Tung, CFO of ASE Holdings, notes that while AI currently in its early-stage and is set to drive explosive growth. As AI integrates into existing and new applications, the demand for advanced packaging is expected to fuel the industry’s entry into the next super growth cycle.

For KYEC, a significant expansion in AI chip testing capacity since Q2 this year positions the company to benefit from the surge in demand.

Sigurd’s COO Tsan-Lien Yeh addresses that, with the release of AI phones, recognizing the doubled testing time for phone chips, which now carry APU/NPU for AI computing compared to general 5G chips. Sigurd has upgraded its equipment to align with future customer needs.

(Image: ASE VIPack’s video)