The recent rapid downturn in the black market prices of AI servers equipped with NVIDIA’s highest-tier AI chip, the H100, in China, has attracted attention, as per a report from Economic Daily News. This fluctuation, triggered by US sanctions, has reportedly prompted concerns about its impact on overall supply and demand dynamics, and whether it will further squeeze normal market mechanisms.

Industry sources cited by the same report have revealed that the prices of AI servers equipped with the H100 chip have recently plummeted on the Chinese black market. This is primarily due to the imminent launch of NVIDIA’s next-generation high-end AI chip, the H200. With the transition between old and new products, scalpers who previously hoarded H100 chips to drive up prices are now offloading their large inventories.

As per a report from Reuters, despite the US expanding its ban on AI technology-related exports to China last year, some dealers are still taking risks. There is still trading of H100 chips in the Huaqiangbei electronics market in northern Shenzhen, but it has all gone underground. The chips are said to be mainly imported into China through purchasing agents or shell companies set up overseas, making them accessible to Chinese universities, research institutions, and even companies through special dealer channels.

Due to the US ban, both the H100 chip and AI servers equipped with it can only be traded on the black market, not openly. Scalpers have significantly inflated prices, with servers featuring the H100 chip reaching over CNY 3 million (over USD 420,000) in China, compared to the official price of USD 280,000 to USD 300,000, resulting in profits of over 10% for some middlemen after deducting logistics and tariffs.

With the H200 set to launch in the second quarter, the H100 will become the “previous generation” product. Consequently, middlemen who had hoarded H100 chips are eager to sell their inventory, leading to a rapid correction in prices.

Recently, servers with the H100 chip on the China black market have dropped to around CNY 2.7 to 2.8 million, with spot prices in Hong Kong falling to around CNY 2.6 million, representing a decline of over 10%.

According to a previous report from Reuters, in response to Chinese universities and research institutions reportedly acquired high-end AI chips from NVIDIA through distributors, a NVIDIA spokesperson stated that the report does not imply that NVIDIA or any of its partners violated export control regulations, and the proportion of these products in global sales is negligible. Nvidia complies with U.S. regulatory standards.

Read more

• [News] NVIDIA’s H100 Chip Heat Declines, CoWoS Capacity Eases
• [News] NVIDIA’s H100 AI Chip No Longer Out of Reach, Inventory Pressure Reportedly Forces Customers to Resell

(Photo credit: NVIDIA)

NVIDIA’s upcoming next-generation high-end AI chip, the H200, is on the horizon. As per a report from Economic Daily News, currently, the mainstream high-end H100 chip has seen a decline in demand, putting an end to the previous state of supply shortages.

As per the same report, Taiwanese contract manufacturers openly acknowledge that the supply of H100 chips is indeed smoother now, primarily due to the alleviation of tight CoWoS advanced packaging capacity constraints.

Despite a significant short-term correction in the market price of H100 chips in China, Taiwan’s AI server manufacturers, such as Quanta and Inventec, are still striving to maximize shipments. This quarter, the momentum of AI server shipments is expected to see a significant boost.

From the perspective of server manufacturers, the demand and pricing of critical components are typically negotiated directly between cloud service providers (CSPs) and chip manufacturers like NVIDIA. Once the price and quantity are agreed upon, manufacturers are then commissioned to produce and ship the products.

Quanta emphasized that with the easing of tight capacity in upstream CoWoS advanced packaging, the supply of H100 chips has become smoother. Maintaining their previous stance, they anticipate that the momentum of AI server shipments will begin to show from this quarter onwards.

A previous report from tom’s hardware once emphasized that the ease of purchasing H100 GPUs has brought about some changes in the market as well. Customers now prioritize price and practicality when leasing AI computing services from cloud service providers.

Additionally, alternatives to the H100 GPU have emerged in the current market, offering comparable performance and software support. These may come at more affordable prices, potentially fostering a fairer market environment.

Mike Yang, Senior Vice President and General Manager of Quanta Cloud Technology (QCT), also mentioned recently that they expect to see a significant improvement in chip supply by June, which will subsequently boost server shipment performance in the second half of the year.

Read more

• [News] NVIDIA’s H100 AI Chip No Longer Out of Reach, Inventory Pressure Reportedly Forces Customers to Resell
• [News] CoWoS Capacity Shortage Challenges AI Chip Demand, while Taiwanese Manufacturers Expand to Seize Opportunities

(Photo credit: NVIDIA)

According to a Reuters, despite the U.S. expanding export controls on advanced artificial intelligence (AI) chips to China last year, Chinese universities and research institutions have recently acquired high-end AI chips from Nvidia through distributors.

Reviewing hundreds of bidding documents, Reuters found that since the U.S. expanded chip export controls on November 17 last year, ten Chinese entities have acquired Nvidia’s advanced chips embedded in server products produced by U.S. firms Supermicro, Dell, and Taiwanese company Gigabyte Technology.

Based on this Reuters report, bidding documents not reported from November 20 last year to February 28 this year show that Chinese institutions such as the Chinese Academy of Sciences, Shandong Artificial Intelligence Institute, Hubei Earthquake Administration, Shandong University, Southwest University, a technology investment company owned by the Heilongjiang Provincial Government, a state-owned aerospace research center, and a space science center have purchased these server products from distributors, which include some of Nvidia’s most advanced chips.

In response, a Nvidia spokesperson told Reuters that the products involved in these bids were exported before the ban was implemented in the United States. The spokesperson stated that the report does not imply that Nvidia or any of its partners violated export control regulations, and the proportion of these products in global sales is negligible. Nvidia complies with U.S. regulatory standards.

Both Supermicro and Dell stated that they would investigate and take action if any third-party illegal exports or re-exports are found. Gigabyte, the Taiwanese company mentioned in the report, told the Central News Agency that it has fully complied with relevant regulations since the chip ban took effect on November 17 last year, and has not shipped any restricted products to China. Gigabyte reiterated its strict adherence to relevant Taiwanese laws and international embargo regulations, stating that there has been no violation of any embargo regulations.

In 2023, the United States further restricted Chinese businesses from acquiring high-end AI chips. At that time, Nvidia responded by launching a China-specific version, the H20. TrendForce also presented relevant data for the Chinese market, indicating that Chinese CSP companies, including ByteDance, Baidu, Alibaba, and Tencent (BBAT), accounted for approximately 6.3% of high-end AI server shipments in 2023. Considering the ban and subsequent risks, it is estimated that the proportion in 2024 may be less than 4%.

(Photo credit: NVIDIA)

Japanese digital infrastructure service provider Sakura Internet, backed by government subsidies, is enhancing its cloud services for generative AI. According to a report from MoneyDJ, Sakura Internet’s procurement of GPUs is set to increase fivefold from the initial plan, with purchases including NVIDIA’s latest product, the “B200,” unveiled in March.

On April 19th, Sakura announced that it has secured Japanese government subsidies to strengthen its cloud service “Koukaryoku” for generative AI. The company plans to expand the number of GPUs deployed in “Koukaryoku” to fivefold from the initially planned quantity, aiming to incorporate around 10,000 GPUs, including NVIDIA’s latest “NVIDIA HGX B200 system” introduced in March. The goal is to establish a large-scale cloud infrastructure with a computational power of 18.9 EFLOPS by the end of March 2028.

Sakura had previously received similar government subsidies in June 2023, marking this as the second time they have received such support.

Sakura announced that last June they invested JPY 13 billion, aiming to purchase approximately 2,000 NVIDIA GPUs (with a computational power of 2.0 EFLOPS) between July 2023 and March 2025. Due to significantly higher demand than expected, the procurement of these 2,000 GPUs is projected to be completed ahead of schedule by the end of June 2024.

This new investment plan, totaling around JPY 100 billion (including costs for server components other than GPUs and maintenance fees), targets additional procurement of approximately 8,000 GPUs (with a computational power of 16.9 EFLOPS) between April 2024 and December 2027.

The overall GPU procurement quantity of around 10,000 units will thus be five times the original plan of approximately 2,000 units. According to Japanese media reports, Sakura will provide server computing power equipped with these GPUs to companies engaged in generative AI research.

On April 19, the Ministry of Economy, Trade and Industry of Japan announced that in order to establish the necessary supercomputers for developing generative AI domestically in Japan, they will provide a maximum subsidy of JPY 72.5 billion to five Japanese companies, with Sakura receiving a maximum subsidy of JPY 50.1 billion.

According to sources cited by the American news outlet Business Insider, Microsoft plans to double its inventory of GPUs to 1.8 million, primarily sourced from NVIDIA. Having more chips on hand will enable Microsoft to launch AI products that are more efficient, faster, and more cost-effective.

The source does not detail specific future applications for these chips, but acquiring a large quantity of chips means that Microsoft can extensively deploy them across its own products, including cloud services and consumer electronics.

The sources cited by the same report further revealed that Microsoft plans to invest USD 100 billion in GPUs and data centers by 2027 to strengthen its existing infrastructure.

Microsoft’s significant stockpiling of AI chips underscores the company’s efforts to maintain a competitive edge in the AI field, where having robust computing power is crucial for innovation.

On the other hand, NVIDIA recently stated that the AI computer they are collaborating on with Microsoft will operate on Microsoft’s Azure cloud platform and will utilize tens of thousands of NVIDIA GPUs, including their H100 and A100 chips.

NVIDIA declined to disclose the contract value of this collaboration. However, industry sources cited by the report estimate that the price of each A100 chip ranges from USD 10,000 to 12,000, while the price of the H100 is significantly higher than this range.

Additionally, Microsoft is also in the process of designing the next generation of the chip. Not only is Microsoft striving to reduce its reliance on NVIDIA, but other companies including OpenAI, Tesla, Google, Amazon, and Meta are also investing in developing their own AI accelerator chips. These companies are expected to compete with NVIDIA’s flagship H100 AI accelerator chips.

Read more

• [News] TSMC Q1 2024 Earnings Call Highlighted Two Major Revisions
• [News] Taiwan Hit by 7.2 Magnitude Earthquake: TSMC and Other Semiconductor, Panel Supply Chain Updates

(Photo credit: NVIDIA)