According to a news report from IJIWEI, sources have revealed that NVIDIA has placed urgent orders with TSMC for the production of AI GPU destined for China. These orders fall under the category of “Super Hot Run” (SHR), with plans to commence fulfillment in the first quarter of 2024.

Respond to the United States implementing stricter export controls on the Chinese semiconductor industry, sources stated in the report indicate that NVIDIA plans to provide a new “specialized” AI chip to China by lowering specifications, replacing the export-restricted H800, A800, and L40S series.

Insiders suggest that NVIDIA intends to resume supplying the RTX 4090 chip to China in January of next year but also release a modified version later to comply with U.S. export restrictions. 

On the other hand, NVIDIA continues to increase its orders with TSMC. This move aims to secure TSMC’s manufacturing capacity to meet the demand for the H100. However, due to limitations in CoWoS (Chip-on-Wafer-on-Substrate) production capacity, the H100 GPU is currently facing severe shortages.

It is noted that following NVIDIA, Intel and AMD are also expected to tailor AI chips for China. TSMC, as the primary pure-play foundry partner for these AI chip suppliers, will continue to enjoy a competitive advantage.

According to sources from semiconductor equipment manufacturers, despite TSMC’s efforts to increase CoWoS production capacity, the foundry still cannot meet the growing demand for NVIDIA GPUs. Additionally, the MI300 chip that was recently launched by AMD is also competing for the foundry industry’s production capacity.

Insiders note that TSMC’s ability to expand CoWoS production capacity is limited, with delays in equipment replacement speed, machine installation speed, and labor deployment. The new capacity is expected to be ready by the second quarter of 2024.

Equipment is identified as one of the key variables affecting TSMC’s expansion of CoWoS production capacity. Unexpected impacts on production and delivery times from Japanese equipment supplier Shibaura have delayed the development and installation of new capacity across TSMC’s production lines, including those in Longtan and Zhunan.

TSMC Chairman Mark Liu mentioned in a press conference in September that the shortage of CoWoS packaging capacity at TSMC is temporary, and it will be addressed through capacity expansion within the next year and a half to meet the growing demand.

(Photo credit: TSMC)

Read more

• [News] TSMC’s CoWoS Demand Surges from NVIDIA, Apple, AMD, Broadcom, Marvell, Monthly Capacity Up 120% in 2024

According to icsmart’s report, in October, Samsung Electronics and SK Hynix received an indefinite exemption from the U.S. government’s control over exports to China, allowing them to import semiconductor manufacturing equipment without special permission for their facilities. Samsung has initiated efforts to boost capacity at its Xi’an plant.

Reportedly, Samsung Electronics‘s 12-inch NAND flash M-FAB fab has officially entered the main construction phase.

Established in Xi’an in 2012, it is Samsung’s sole overseas memory production base and has evolved into the world’s largest NAND Flash manufacturing facility. The fab produces over 265,000 12-inch wafers per month, contributing to over 40% of Samsung’s total NAND Flash flash production.

According to publicly available data, the initial investment for the first phase of Samsung’s Xi’an fab in China was USD 10.87 billion, and it commenced production in May 2014, primarily manufacturing 3D NAND flash memory chips.

On August 30, 2017, Samsung Semiconductor announced a USD 7 billion investment to build the second phase of the 12-inch NAND flash project, establishing a new NAND flash production line. In December 2019, the company decided to further invest USD 8 billion to expand the scale of the second-phase project.

Business Korea reports that Samsung executives have decided to upgrade the Xi’an NAND Flash fab to a 236-layer stacking process and significantly expand production. Industry sources indicate that Samsung has initiated the procurement of semiconductor equipment, with deliveries scheduled for the end of the year.

In 2024, the company plans to introduce eighth-generation NAND Flash equipment in succession. This move is seen by the industry as a strategy to counter the soft demand in the global NAND Flash market.

(Photo credit: Samsung)

On November 22, 2023, NVIDIA released its financial report for the third quarter of 2023 (FY3Q24: August 2023 to October 2023), with a revenue of USD 18.1 billion. This represents a quarterly increase of 34% and a yearly increase of 206%.

NVIDIA also provided a revenue guidance for the fourth quarter of 2023 (FY4Q24: November 2023 to January 2024), with a median estimate of USD 20 billion. This reflects a quarterly increase of 10.5% and a yearly increase of 231%.

TrendForce’s Insight:

1. Downgraded Chips Expected by Year-End 

NVIDIA continues its robust performance in the third quarter of 2023 (FY3Q24). The Datacenter division reported a revenue of USD 14.5 billion, marking a 41% quarterly increase and a staggering 279% annual increase. This segment now constitutes 80% of the overall revenue, with a 4% increase from the previous quarter.

The growth is primarily driven by the HGX Hopper GPU, along with the commencement of shipments and revenue recognition for L40S and GH200. Approximately 50% of Datacenter revenue comes from CSP customers, while the remaining 50% is contributed by Consumer Internet and enterprise clients.

In terms of revenue outlook, the China region accounts for approximately 20-25% of NVIDIA’s Datacenter revenue. The management acknowledges the significant impact of the U.S. restrictions on China’s revenue for the fourth quarter of 2023 but expresses confidence that revenue from other regions can offset this impact.

This confidence stems from the current high demand and low supply situation for AI chips. Notably, NVIDIA’s anticipated release of lower-capacity chips, including HGX H20, L20 PCIe, and L2PCIe, originally slated for November 16, 2023, is now expected to be delayed until the end of 2023, presumably due to ongoing negotiations with the U.S. Department of Commerce.

2. NVIDIA Unveils HGX H200, High-Capacity Version of H100; Next-Gen B100 Anticipates Doubling Performance

In a significant product announcement, NVIDIA introduced HGX H200 on November 14, 2023. The new offering is a high-capacity version of H100 and is fully compatible with HGX H100 systems. This compatibility ensures that manufacturers using H100 don’t need to modify server systems or software specifications when transitioning to H200.

HGX H200 is slated for release in the second quarter of 2024, with initial customers including AWS, Microsoft Azure, Google Cloud, and Oracle Cloud. Additionally, CoreWeave, Lambda, and Vultr are expected to adopt HGX H200 in their setups.

Comparing H200 SXM with H100 SXM, it’s evident that H200 SXM has a 76% increase in memory capacity and a 43% increase in bandwidth compared to H100 SXM. Additionally, it upgrades from HBM3 to HBM3e, while the remaining specifications remain the same. This indicates that H200 SXM is essentially a high-capacity version of H100 SXM.

Given the sensitivity of performance to memory capacity and bandwidth, inference on the Llama2 with 7 billion parameters shows that the performance of H200 SXM can reach 1.9 times that of H100 SXM.

Moreover, NVIDIA plans to launch the B100, truly representing the next generation of products with the new Blackwell architecture in 2024. Utilizing Chiplet technology, it is speculated that the architecture will transition from a single die + 6 HBM3 configuration in H100 to a dual-die + 8 HBM3e configuration in B100, potentially doubling the performance compared to H100.

Read more

• [News] H200 Unveiled: NVIDIA Integrates HBM3e for Enhanced AI Performance

(Photo credit: NVIDIA)

On October 17, 2023, the U.S. government unveiled an updated set of regulations for semiconductor exports, introducing stricter standards for advanced AI chips. Additionally, these regulations expand control over the export of exposure equipment and include Chinese GPU design startups on an Entity List.

TrendForce’s Insights: 

1. New regulations cover chips, manufacturing equipment, and related companies, signaling an effort to restrain China’s AI development.

In this latest set of regulations, the U.S. has relaxed the I/O bandwidth restrictions for AI chips and introduced three additional conditions beyond a total processing performance (TPP) of ≥ 4800 TOPS:

(1) Total processing performance ≥ 1600 TOPS and performance density (PD) ≥ 5.92

(2) Total processing performance ≥ 2400 TOPS but < 4800 TOPS and performance density ≥ 1.6 but < 5.92

(3) Total processing performance ≥ 1600 TOPS and performance density ≥ 3.2 but < 5.92

As a result of these new conditions, NVIDIA’s A800, H800 GPU, and the recent launched L40S GPU for the Chinese market are now included in the list of controlled exports, similar to the A100 and H100 GPUs that were added in September 2022.

Concerning manufacturing equipment, the control threshold for exposure equipment has shifted from single-machine (specified substrate) coverage precision of ≤ 1.5nm to > 1.5nm but ≤ 2.4nm. This change directly led to the inclusion of ASML’s 1980Di DUV lithography machines.

On the corporate front, Chinese domestic GPU design startups such as Birentech, Moore Threads, and high-speed DSP design company Superfusion Semiconductor, along with their related entities, have been placed on the Entity List by the U.S. Department of Commerce.

In summary, these new regulations encompass chips, manufacturing equipment, and related companies. The U.S. is not only controlling the current mainstream AI product lines and applications of DUV lithography machines for 28-7nm processes but is also making a clear effort to interfere Chinese domestic manufacturers’ development of AI computation chips, indicating a strong determination to restrict China’s growth in the AI sector.

2. Priority for Chinese Enterprises: Securing AI Computing Resources through Cloud Service Providers

In light of the impact of the new U.S. semiconductor control regulations, Chinese domestic companies will be limited to AI chip performance not exceeding that of NVIDIA L40 GPU. As leading companies like NVIDIA, AMD, Intel, and others continuously boost the performance of their AI chips, the gap between the AI computing resources established by Chinese companies and their international counterparts will continue to widen.

Looking at it from an angle of independent research and development, with the inclusion of 1980Di and more advanced DUV lithography machines in the control list and the U.S. Department of Commerce placing Chinese IC design companies on the Entity List, short-term mass production of high-performance server AI chips in China seems unlikely.

Faced with challenges in both outsourcing and in-house production, the primary path for Chinese domestic companies to develop AI technology and applications is to obtain high-performance AI computing resources from international cloud service providers (CSP). It is worth noting that the U.S. government is also exploring limitations on Chinese firms attempting to evade semiconductor control policies through CSP. For Chinese companies, establishing robust customer relationships and building extensive AI computing resources are pressing priorities before related policies are enacted.
(Image: Pixabay)