Despite U.S. export controls aimed at preventing Chinese companies from acquiring advanced AI chips, small cloud service providers in China have reportedly found ways to obtain NVIDIA’s A100 and H100 chips. The cost of renting cloud services in China is even lower than in the U.S.

According to a report from the Financial Times, four small-scale Chinese cloud providers are offering servers equipped with eight A100 chips each, charging around USD 6 per hour. In comparison, similar services from U.S. cloud providers cost approximately USD 10 per hour.

As Chinese companies are reportedly bypassing U.S. export controls, industry sources cited by the Financial Times have further noted that the lower prices in China may hint at a robust local supply of NVIDIA chips.

Since the fall of 2022, the U.S. has banned NVIDIA from supplying A100 chips to China, and the more powerful H100 chips have not been approved for sale there.

However, industry sources and startups have revealed that these chips are still available in China. Ads for A100 and H100 have appeared on social media platforms like Xiaohongshu and e-commerce sites such as Taobao, with prices higher than those abroad.

At the Huaqiangbei electronics market in Shenzhen, reportedly, industry sources have revealed that the price of NVIDIA’s H100 is quoted at USD 23,000 to USD 30,000, while Chinese online sellers list it at USD 31,000 to USD 33,000.

Meanwhile, larger Chinese cloud providers such as Alibaba and ByteDance emphasize service stability and security in the local market. For servers equipped with A100 chips, they charge two to four times more than smaller cloud providers.

According to another source cited by Financial Times, large companies must consider regulatory compliance, which puts them at a disadvantage because they are reluctant to use smuggled chips. In contrast, smaller providers are less concerned.

The same report also indicate that after the US government tightened export controls in October last year, servers from Supermicro equipped with eight H100 chips were priced as high as approximately CNY 3.2 million. However, as supply constraints eased, the price has dropped to around CNY 2.5 million.

Several sources cited by the report claim that merchants from Malaysia, Japan, and Indonesia frequently ship Supermicro servers or NVIDIA chips to Hong Kong, from where they are then transported to Shenzhen.

In response to these issues, NVIDIA reportedly stated that it primarily sells chips to well-known partners, ensuring that all sales comply with U.S. export regulations.

NVIDIA also mentioned that its used products can be obtained through various channels and, although they cannot track products after sale, they will take appropriate action if they determine a customer is violating U.S. export controls.

Read more

• [News] China’s 1H24 Chip Equipment Purchases Exceed Taiwan, Korea, and US Combined, Reaching USD 25 Billion
• [News] China Contributes Over 40% of U.S. Chip Equipment Makers’ Revenue despite Export Controls

(Photo credit: iStock)

The US Department of Commerce issued new restrictions on AI chips on October 17, 2023, with a focus on controlling the export of chips to China, including NIVIDA’s A800, H800, L40S, and RTX4090, among others. Taiwanese manufacturers primarily serve cloud service providers and brand owners in North America, with relatively fewer shipments to Chinese servers. However, Chinese manufacturers, having already faced two chip restrictions imposed by the US, recognize the significance of AI chips in server applications and are expected to accelerate their in-house chip development processes.

TrendForce’s Insights:

1. Limited Impact on Taiwanese Manufacturers in Shipping AI Servers with H100 GPUs

Major Taiwanese server manufacturering companies, including Foxconn, Quanta, Inventec, GIGABYTE, and Wiwynn, provide AI servers equipped with H100 GPUs to cloud data centers and brand owners in Europe and the United States. These Taiwanese companies have established some AI server factories outside China, in countries such as the US, the Czech Republic, Mexico, Malaysia, and Thailand, focusing on producing L10 server units and L11 cabinets in proximity to end-users. This strategy aligns with the strategic needs of US cloud providers and brand owners for global server product deployment.

On the other hand, including MiTAC, Wistron, and Inventec, also provide server assembly services for Chinese brands such as Inspur and Lenovo. Although MiTAC has a significant share in assembling Inspur’s servers, it acquired Intel DSG (Data Center Solutions Group) business in July 2023. Therefore, the focus of AI servers remains on brand manufacturers using H100 GPUs, including Twitter, Dell, AWS, and European cloud service provider OVH. It is speculated that the production ratio of brand servers will be adjusted before the new restrictions are enforced.

Wistron is a major supplier for NVIDIA’s AI server modules, DGX A100, and HGX H100. Its primary shipments are to end-users in Europe and the United States. It is expected that there will be adjustments in the proportion of shipments to Chinese servers following the implementation of the restrictions.

Compal has fewer AI server orders compared to other Taiwanese manufacturers. It has not yet manifested any noticeable changes in Lenovo server assembly proportions. The full extent of the impact will only become more apparent after the enforcement of the ban.

During the transitional period before the implementation of the chip ban in the United States, the server supply chain can still adapt shipments based on local chip demand in China to address market impacts resulting from subsequent chip controls.

2. Chinese Manufacturers Focusing on Accelerating In-House Chip Development

Chinese cloud companies had already started developing their AI chips before the first U.S. chip restrictions in 2022. This included self-developed AI chips like Alibaba Cloud’s T-HEAD, a data center AI chip, and they expanded investments in areas such as DRAM, AI chips, and semiconductors with the aim of establishing a comprehensive IoT system from chips to the cloud.

Baidu Cloud, on the other hand, accelerated the development of its third-generation self-developed Kunlun chip, designed for cloud and edge computing, with plans for an early 2024 release.

Tencent introduced three self-developed chips in 2021, including an AI inference chip called Zixiao, used for Tencent’s meeting business; a video transcoding chip called Canghai, used in cloud gaming and live streaming applications; and a smart network card chip named Xuanling, applied in network storage and computing.

ByteDance made investments in cloud AI chips through its MooreThread initiative in 2022 for applications in AI servers. Huawei released the Ascend 900 chip in 2019 and is expected to introduce the Ascend 930B AI chip in the latter half of 2024. While this chip has the same computational power as the NVIDIA A100 chip, its performance still requires product validation, and it is speculated that it may not replace the current use of NVIDIA GPUs in Chinese AI servers.

Despite the acceleration of self-developed chip development among Chinese cloud server manufacturers, the high technological threshold, lengthy development cycles, and high costs associated with GPU development often delay the introduction of new server products. Therefore, Chinese cloud companies and brand manufacturers continue to purchase NVIDIA GPUs for the production of mid to high-end servers to align with their economic scale and production efficiency.

In response to the new U.S. restrictions, Chinese cloud companies have adopted short-term measures such as increasing imports of existing NVIDIA chips and building up stockpiles before the enforcement of the new restrictions. They are also focusing on medium to long-term strategies, including accelerating resource integration and shortening development timelines to expedite GPU chip manufacturing processes, thus reducing dependency on U.S. restrictions.

In its FY2Q24 earnings report for 2023, NVIDIA disclosed that the U.S. government had imposed controls on its AI chips destined for the Middle East. However, on August 31, 2023, the U.S. Department of Commerce stated that they had “not prohibited the sale of chips to the Middle East” and declined to comment on whether new requirements had been imposed on specific U.S. companies. Both NVIDIA and AMD have not responded to this issue.

TrendForce’s analysis:

• Close ties between Middle Eastern countries and China raise U.S. concerns:

In NVIDIA’s FY2Q24 earnings report, it mentioned, “During the second quarter of fiscal year 2024, the USG informed us of an additional licensing requirement for a subset of A100 and H100 products destined to certain customers and other regions, including some countries in the Middle East.” It is speculated that the U.S. is trying to prevent high-speed AI chips from flowing into the Chinese market via the Middle East. This has led to controls on the export of AI chips to the Middle East.

Since August 2022, the U.S. has imposed controls on NVIDIA A100, H100, AMD MI100, MI200, and other AI-related GPUs, restricting the export of AI chips with bidirectional transfer rates exceeding 600GB/s to China. Saudi Arabia had already signed a strategic partnership with China in 2022 for cooperation in the digital economy sector, including AI, advanced computing, and quantum computing technologies. Additionally, the United Arab Emirates has expressed interest in AI cooperation with China. There have been recent reports of Saudi Arabia heavily acquiring NVIDIA’s AI chips, which has raised concerns in the U.S.

• Huawei is expected to release AI chips comparable to NVIDIA A100 in the second half of 2024; competition is yet to be observed:

Affected by U.S. sanctions, Chinese companies are vigorously developing AI chips. iFlytek is planning to launch a new general-purpose LLM (Large Language Model) in October 2023, and the AI chip Ascend 910B, co-developed with Huawei, is expected to hit the market in the second half of 2024, with performance claimed to rival that of NVIDIA A100. In fact, Huawei had already introduced the Ascend 910, which matched the performance of NVIDIA’s V100, in 2019. Considering Huawei’s Kirin 9000s, featured in the flagship smartphone Mate 60 Pro released in August 2023, it is highly likely that Huawei can produce products with performance comparable to A100.

However, it’s important to note that the A100 was already announced by NVIDIA in 2020. This means that even if Huawei successfully launches a new AI chip, it will already be four years behind NVIDIA. Given the expected 7nm process for Huawei’s Ascend 910B and NVIDIA’s plan to release the 3nm process-based Blackwell architecture GPU B100 in the second half of 2024, Huawei will also lag behind by two generations in chip fabrication technology. With the parameters of LLM doubling annually, the competitiveness of Huawei’s new AI chip remains to be observed.

• China remains NVIDIA’s dominion in the short term:

Despite the active development of AI chips by Chinese IC design house, NVIDIA’s AI chips remain the preferred choice for training LLM models among Chinese cloud companies. Looking at the revenue performance of the leading Chinese AI chip company, Cambricon, its revenue for the first half of 2023 was only CNY 114 million, a YoY decrease of 34%. While being added to the U.S. Entity List was a major reason for the revenue decline, NVIDIA’s dominance in the vast Chinese AI market is also a contributing factor. It is estimated that NVIDIA’s market share in the Chinese GPU market for AI training exceeded 95% in the first half of 2023. In fact, in the second quarter of 2023, the China market accounted for 20-25% of NVIDIA’s Data Center segment revenue.

The main reason for this is that the Chinese AI ecosystem is still quite fragmented and challenging to compete with NVIDIA’s CUDA ecosystem. Therefore, Chinese companies are actively engaged in software development. However, building a sufficiently attractive ecosystem to lure Chinese CSPs in the short term remains quite challenging. Consequently, it is expected that NVIDIA will continue to dominate the Chinese market for the next 2-3 years.

(Photo credit: NVIDIA)

According to the news from Chinatimes, Asus, a prominent technology company, has announced on the 30th of this month the release of AI servers equipped with NVIDIA’s L40S GPUs. These servers are now available for order. The L40S GPU was introduced by NVIDIA in August to address the shortage of H100 and A100 GPUs. Remarkably, Asus has swiftly responded to this situation by unveiling AI server products within a span of less than two weeks, showcasing their optimism in the imminent surge of AI applications and their eagerness to seize the opportunity.

Solid AI Capabilities of Asus Group

Apart from being among the first manufacturers to introduce the NVIDIA OVX server system, Asus has leveraged resources from its subsidiaries, such as TaiSmart and Asus Cloud, to establish a formidable AI infrastructure. This not only involves in-house innovation like the Large Language Model (LLM) technology but also extends to providing AI computing power and enterprise-level generative AI applications. These strengths position Asus as one of the few all-encompassing providers of generative AI solutions.

Projected Surge in Server Business

Regarding server business performance, Asus envisions a yearly compounded growth rate of at least 40% until 2027, with a goal of achieving a fivefold growth over five years. In particular, the data center server business catering primarily to Cloud Service Providers (CSPs) anticipates a tenfold growth within the same timeframe, driven by the adoption of AI server products.

Asus CEO recently emphasized that Asus’s foray into AI server development was prompt and involved collaboration with NVIDIA from the outset. While the product lineup might be more streamlined compared to other OEM/ODM manufacturers, Asus had secured numerous GPU orders ahead of the AI server demand surge. The company is optimistic about the shipping momentum and order visibility for the new generation of AI servers in the latter half of the year.

Embracing NVIDIA’s Versatile L40S GPU

The NVIDIA L40S GPU, built on the Ada Lovelace architecture, stands out as one of the most powerful general-purpose GPUs in data centers. It offers groundbreaking multi-workload computations for large language model inference, training, graphics, and image processing. Not only does it facilitate rapid hardware solution deployment, but it also holds significance due to the current scarcity of higher-tier H100 and A100 GPUs, which have reached allocation stages. Consequently, businesses seeking to repurpose idle data centers are anticipated to shift their focus toward AI servers featuring the L40S GPU.

Asus’s newly introduced L40S GPU servers include the ESC8000-E11/ESC4000-E11 models with built-in Intel Xeon processors, as well as the ESC8000A-E12/ESC4000A-E12 models utilizing AMD EPYC processors. These servers can be configured with up to 4 or a maximum of 8 NVIDIA L40S GPUs. This configuration assists enterprises in enhancing training, fine-tuning, and inference workloads, facilitating AI model creation. It also establishes Asus’s platforms as the preferred choice for multi-modal generative AI applications.

(Source: https://www.chinatimes.com/newspapers/20230831000158-260202?chdtv)

According to a report from Taiwan’s TechNews, NVIDIA has delivered impressive results in its latest financial report, coupled with an optimistic outlook for its financial projections. This demonstrates that the demand for AI remains robust for the coming quarters. Currently, NVIDIA’s H100 and A100 chips both utilize TSMC’s CoWoS advanced packaging technology, making TSMC’s production capacity a crucial factor.

Examining the core GPU market, NVIDIA holds a dominant market share of 90%, while AMD accounts for about 10%. While other companies might adopt Google’s TPU or develop customized chips, they currently lack significant operational cost advantages.

In the short term, the shortage of CoWoS has led to tight chip supplies. However, according to a recent report by Morgan Stanley Securities, NVIDIA believes that TSMC’s CoWoS capacity won’t restrict shipments of the next quarter’s H100 GPUs. The company anticipates an increase in supply for each quarter next year. Simultaneously, TSMC is raising CoWoS prices by 20% for rush orders, indicating that the anticipated CoWoS bottleneck might alleviate.

According to industry sources, NVIDIA is actively diversifying its CoWoS supply chain away from TSMC. UMC, ASE, Amkor, and SPIL are significant players in this effort. Currently, UMC is expanding its interposer production capacity, aiming to double its capacity to relieve the tight CoWoS supply situation.

According to Morgan Stanley Securities, TSMC’s monthly CoWoS capacity this year is around 11,000 wafers, projected to reach 25,000 wafers by the end of next year. Non-TSMC CoWoS supply chain’s monthly capacity can reach 3,000 wafers, with a planned increase to 5,000 wafers by the end of next year.

(Photo credit: TSMC)