NVIDIA’s upcoming next-generation high-end AI chip, the H200, is on the horizon. As per a report from Economic Daily News, currently, the mainstream high-end H100 chip has seen a decline in demand, putting an end to the previous state of supply shortages.

As per the same report, Taiwanese contract manufacturers openly acknowledge that the supply of H100 chips is indeed smoother now, primarily due to the alleviation of tight CoWoS advanced packaging capacity constraints.

Despite a significant short-term correction in the market price of H100 chips in China, Taiwan’s AI server manufacturers, such as Quanta and Inventec, are still striving to maximize shipments. This quarter, the momentum of AI server shipments is expected to see a significant boost.

From the perspective of server manufacturers, the demand and pricing of critical components are typically negotiated directly between cloud service providers (CSPs) and chip manufacturers like NVIDIA. Once the price and quantity are agreed upon, manufacturers are then commissioned to produce and ship the products.

Quanta emphasized that with the easing of tight capacity in upstream CoWoS advanced packaging, the supply of H100 chips has become smoother. Maintaining their previous stance, they anticipate that the momentum of AI server shipments will begin to show from this quarter onwards.

A previous report from tom’s hardware once emphasized that the ease of purchasing H100 GPUs has brought about some changes in the market as well. Customers now prioritize price and practicality when leasing AI computing services from cloud service providers.

Additionally, alternatives to the H100 GPU have emerged in the current market, offering comparable performance and software support. These may come at more affordable prices, potentially fostering a fairer market environment.

Mike Yang, Senior Vice President and General Manager of Quanta Cloud Technology (QCT), also mentioned recently that they expect to see a significant improvement in chip supply by June, which will subsequently boost server shipment performance in the second half of the year.

Read more

• [News] NVIDIA’s H100 AI Chip No Longer Out of Reach, Inventory Pressure Reportedly Forces Customers to Resell
• [News] CoWoS Capacity Shortage Challenges AI Chip Demand, while Taiwanese Manufacturers Expand to Seize Opportunities

(Photo credit: NVIDIA)

According to a Reuters, despite the U.S. expanding export controls on advanced artificial intelligence (AI) chips to China last year, Chinese universities and research institutions have recently acquired high-end AI chips from Nvidia through distributors.

Reviewing hundreds of bidding documents, Reuters found that since the U.S. expanded chip export controls on November 17 last year, ten Chinese entities have acquired Nvidia’s advanced chips embedded in server products produced by U.S. firms Supermicro, Dell, and Taiwanese company Gigabyte Technology.

Based on this Reuters report, bidding documents not reported from November 20 last year to February 28 this year show that Chinese institutions such as the Chinese Academy of Sciences, Shandong Artificial Intelligence Institute, Hubei Earthquake Administration, Shandong University, Southwest University, a technology investment company owned by the Heilongjiang Provincial Government, a state-owned aerospace research center, and a space science center have purchased these server products from distributors, which include some of Nvidia’s most advanced chips.

In response, a Nvidia spokesperson told Reuters that the products involved in these bids were exported before the ban was implemented in the United States. The spokesperson stated that the report does not imply that Nvidia or any of its partners violated export control regulations, and the proportion of these products in global sales is negligible. Nvidia complies with U.S. regulatory standards.

Both Supermicro and Dell stated that they would investigate and take action if any third-party illegal exports or re-exports are found. Gigabyte, the Taiwanese company mentioned in the report, told the Central News Agency that it has fully complied with relevant regulations since the chip ban took effect on November 17 last year, and has not shipped any restricted products to China. Gigabyte reiterated its strict adherence to relevant Taiwanese laws and international embargo regulations, stating that there has been no violation of any embargo regulations.

In 2023, the United States further restricted Chinese businesses from acquiring high-end AI chips. At that time, Nvidia responded by launching a China-specific version, the H20. TrendForce also presented relevant data for the Chinese market, indicating that Chinese CSP companies, including ByteDance, Baidu, Alibaba, and Tencent (BBAT), accounted for approximately 6.3% of high-end AI server shipments in 2023. Considering the ban and subsequent risks, it is estimated that the proportion in 2024 may be less than 4%.

(Photo credit: NVIDIA)

Japanese digital infrastructure service provider Sakura Internet, backed by government subsidies, is enhancing its cloud services for generative AI. According to a report from MoneyDJ, Sakura Internet’s procurement of GPUs is set to increase fivefold from the initial plan, with purchases including NVIDIA’s latest product, the “B200,” unveiled in March.

On April 19th, Sakura announced that it has secured Japanese government subsidies to strengthen its cloud service “Koukaryoku” for generative AI. The company plans to expand the number of GPUs deployed in “Koukaryoku” to fivefold from the initially planned quantity, aiming to incorporate around 10,000 GPUs, including NVIDIA’s latest “NVIDIA HGX B200 system” introduced in March. The goal is to establish a large-scale cloud infrastructure with a computational power of 18.9 EFLOPS by the end of March 2028.

Sakura had previously received similar government subsidies in June 2023, marking this as the second time they have received such support.

Sakura announced that last June they invested JPY 13 billion, aiming to purchase approximately 2,000 NVIDIA GPUs (with a computational power of 2.0 EFLOPS) between July 2023 and March 2025. Due to significantly higher demand than expected, the procurement of these 2,000 GPUs is projected to be completed ahead of schedule by the end of June 2024.

This new investment plan, totaling around JPY 100 billion (including costs for server components other than GPUs and maintenance fees), targets additional procurement of approximately 8,000 GPUs (with a computational power of 16.9 EFLOPS) between April 2024 and December 2027.

The overall GPU procurement quantity of around 10,000 units will thus be five times the original plan of approximately 2,000 units. According to Japanese media reports, Sakura will provide server computing power equipped with these GPUs to companies engaged in generative AI research.

On April 19, the Ministry of Economy, Trade and Industry of Japan announced that in order to establish the necessary supercomputers for developing generative AI domestically in Japan, they will provide a maximum subsidy of JPY 72.5 billion to five Japanese companies, with Sakura receiving a maximum subsidy of JPY 50.1 billion.

According to sources cited by the American news outlet Business Insider, Microsoft plans to double its inventory of GPUs to 1.8 million, primarily sourced from NVIDIA. Having more chips on hand will enable Microsoft to launch AI products that are more efficient, faster, and more cost-effective.

The source does not detail specific future applications for these chips, but acquiring a large quantity of chips means that Microsoft can extensively deploy them across its own products, including cloud services and consumer electronics.

The sources cited by the same report further revealed that Microsoft plans to invest USD 100 billion in GPUs and data centers by 2027 to strengthen its existing infrastructure.

Microsoft’s significant stockpiling of AI chips underscores the company’s efforts to maintain a competitive edge in the AI field, where having robust computing power is crucial for innovation.

On the other hand, NVIDIA recently stated that the AI computer they are collaborating on with Microsoft will operate on Microsoft’s Azure cloud platform and will utilize tens of thousands of NVIDIA GPUs, including their H100 and A100 chips.

NVIDIA declined to disclose the contract value of this collaboration. However, industry sources cited by the report estimate that the price of each A100 chip ranges from USD 10,000 to 12,000, while the price of the H100 is significantly higher than this range.

Additionally, Microsoft is also in the process of designing the next generation of the chip. Not only is Microsoft striving to reduce its reliance on NVIDIA, but other companies including OpenAI, Tesla, Google, Amazon, and Meta are also investing in developing their own AI accelerator chips. These companies are expected to compete with NVIDIA’s flagship H100 AI accelerator chips.

Read more

• [News] TSMC Q1 2024 Earnings Call Highlighted Two Major Revisions
• [News] Taiwan Hit by 7.2 Magnitude Earthquake: TSMC and Other Semiconductor, Panel Supply Chain Updates

(Photo credit: NVIDIA)

The US Department of Commerce has added leading Chinese solution provider Sitonholy, who primarily sells processors from Intel and NVIDIA, to its Entity List. This inclusion on the Entity List undoubtedly impacts Sitonholy but also has significant implications for American companies like Intel and NVIDIA.

According to reports from the South China Morning Post and Reuters, Sitonholy sells hardware based on Intel and NVIDIA technologies and also provides cloud services. As a result, US companies engaging in business with Sitonholy require export licenses from the Department of Commerce, but these license applications are presumed to be denied during the review process.

This development is a significant blow to Sitonholy and American companies alike. Kevin Kurland, a US export enforcement official, stated during a hearing of the US Senate subcommittee that the US government has placed four Chinese companies on an export blacklist for assisting the Chinese military in obtaining AI chips. The four Chinese companies are Linkzol Technology, Xi’an Like Innovative Information Technology, Beijing Anwise Technology, and Sitonholy.

A Chinese Foreign Ministry spokesperson criticized the United States for unfairly targeting Chinese companies through export controls and demanded that the US stop politicizing trade and technology issues.

Read more

• [News] US Government Considers Adding ChangXin Memory Technologies to Entity List, Imposing Further Sanctions on Chinese Firms
• [News] Bypassing U.S. Restrictions? CXMT Showcases GAA Technology for 3nm Chips
• DRAM Industry Sees Nearly 30% Revenue Growth in 4Q23 Due to Rising Prices and Volume, Says TrendForce

(Photo credit: iStock)