According to sources cited by the American news outlet Business Insider, Microsoft plans to double its inventory of GPUs to 1.8 million, primarily sourced from NVIDIA. Having more chips on hand will enable Microsoft to launch AI products that are more efficient, faster, and more cost-effective.

The source does not detail specific future applications for these chips, but acquiring a large quantity of chips means that Microsoft can extensively deploy them across its own products, including cloud services and consumer electronics.

The sources cited by the same report further revealed that Microsoft plans to invest USD 100 billion in GPUs and data centers by 2027 to strengthen its existing infrastructure.

Microsoft’s significant stockpiling of AI chips underscores the company’s efforts to maintain a competitive edge in the AI field, where having robust computing power is crucial for innovation.

On the other hand, NVIDIA recently stated that the AI computer they are collaborating on with Microsoft will operate on Microsoft’s Azure cloud platform and will utilize tens of thousands of NVIDIA GPUs, including their H100 and A100 chips.

NVIDIA declined to disclose the contract value of this collaboration. However, industry sources cited by the report estimate that the price of each A100 chip ranges from USD 10,000 to 12,000, while the price of the H100 is significantly higher than this range.

Additionally, Microsoft is also in the process of designing the next generation of the chip. Not only is Microsoft striving to reduce its reliance on NVIDIA, but other companies including OpenAI, Tesla, Google, Amazon, and Meta are also investing in developing their own AI accelerator chips. These companies are expected to compete with NVIDIA’s flagship H100 AI accelerator chips.

Read more

• [News] TSMC Q1 2024 Earnings Call Highlighted Two Major Revisions
• [News] Taiwan Hit by 7.2 Magnitude Earthquake: TSMC and Other Semiconductor, Panel Supply Chain Updates

(Photo credit: NVIDIA)

The US Department of Commerce has added leading Chinese solution provider Sitonholy, who primarily sells processors from Intel and NVIDIA, to its Entity List. This inclusion on the Entity List undoubtedly impacts Sitonholy but also has significant implications for American companies like Intel and NVIDIA.

According to reports from the South China Morning Post and Reuters, Sitonholy sells hardware based on Intel and NVIDIA technologies and also provides cloud services. As a result, US companies engaging in business with Sitonholy require export licenses from the Department of Commerce, but these license applications are presumed to be denied during the review process.

This development is a significant blow to Sitonholy and American companies alike. Kevin Kurland, a US export enforcement official, stated during a hearing of the US Senate subcommittee that the US government has placed four Chinese companies on an export blacklist for assisting the Chinese military in obtaining AI chips. The four Chinese companies are Linkzol Technology, Xi’an Like Innovative Information Technology, Beijing Anwise Technology, and Sitonholy.

A Chinese Foreign Ministry spokesperson criticized the United States for unfairly targeting Chinese companies through export controls and demanded that the US stop politicizing trade and technology issues.

Read more

• [News] US Government Considers Adding ChangXin Memory Technologies to Entity List, Imposing Further Sanctions on Chinese Firms
• [News] Bypassing U.S. Restrictions? CXMT Showcases GAA Technology for 3nm Chips
• DRAM Industry Sees Nearly 30% Revenue Growth in 4Q23 Due to Rising Prices and Volume, Says TrendForce

(Photo credit: iStock)

Under pressure from US restrictions, Intel is reportedly preparing to follow in NVIDIA’s footsteps by developing “special edition” versions of its AI acceleration chips, Gaudi 3, for the Chinese market. These two related products are rumored to be launched at the end of June and the end of September.

According to reports from The Register, Intel recently unveiled its new generation AI acceleration chip, Gaudi 3. Intel stated in the Gaudi 3 white paper that it is preparing to launch a special edition Gaudi 3 tailored for the Chinese market. This would include two hardware variants: the HL-328 OAM-compatible Mezzanine Card and the HL-388 PCIe Accelerator Card. The HL-328 is said to be scheduled for release on June 24, while the HL-388 follow suit on September 24.

In regard of the specifications, the made-for-China edition and the original version share the same features, including 96MB of on-chip SRAM memory, 128GB of HBM2e high-bandwidth memory with a bandwidth of 3.7TB per second, PCIe 5.0X16 interface, and decoding standards.

However, due to US export restrictions on AI chips, the comprehensive computing performance (TPP) of high-performance AI needs to be below 4,800 to export to China. This means the Chinese special edition’s 16-bit performance cannot exceed 150 TFLOPS (trillion floating-point operations per second).

For comparison, the original Gaudi 3 achieves 1,835 TFLOPS in FP16/BF16. This contrasts with NVIDIA’s H100, which is approximately 40% faster in large model training and 50% more efficient in inference tasks.

Therefore, the made-for-China edition will need to significantly reduce the number of cores (the original version has 8 Matrix Multiplication Engines [MME] and 64 Tensor Processor Core [TPC] engines) and operating frequency. Ultimately, this could result in reducing its AI performance by approximately 92% to comply with US export control requirements.

Analyses cited in the same report further suggest that Intel’s launch of the made-for-China edition for AI performance will be comparable to NVIDIA’s AI accelerator card H20 tailored for the Chinese market.

The made-for-China edition of Intel’s Gaudi 3 boasts a performance of 148 TFLOPS in FP16/BF16, slightly below the 150 TFLOPS limit. However, in terms of high-bandwidth memory (HBM) capacity and bandwidth, the Chinese special edition Gaudi 3 will be lower than NVIDIA’s H20, potentially putting it at a competitive disadvantage against the H20. Still, pricing will also be a key factor in determining whether it holds any competitive advantage.

As per a previous report from Reuters, the prices of the chips were said to be comparable to those of its competitor Huawei’s products. Reportedly, NVIDIA priced orders from Chinese H20 distributors between USD 12,000 and 15,000 per unit.

TrendForce believes Chinese companies will continue to buy existing AI chips in the short term. NVIDIA’s GPU AI accelerator chips remain a top priority—including H20, L20, and L2—designed specifically for the Chinese market following the ban.

Read more

• [News] Expert Insights on NVIDIA’s AI Chip Strategy – Downgraded Version Targeted for China, High-End Versions Aimed Overseas
• NVIDIA Experiences Strong Cloud AI Demand but Faces Challenges in China, with High-End AI Server Shipments Expected to Be Below 4% in 2024, Says TrendForce

(Photo credit: NVIDIA)

Meta Platform, parent company of Facebook, has announced its latest generation AI chip of its Training and Inference Accelerator (MTIA) on April 10th, fabricated using TSMC’s 5nm process. According to a report from Commercial Times, this move is expected to reduce Meta’s reliance on NVIDIA’s chips and enhance computational power for AI services.

In its shift towards AI services, Meta requires greater computational capabilities. Thus, last year, Meta introduced its AI models to compete with OpenAI’s ChatGPT. The latest AI chip, Artemis, is an upgraded version of MTIA introduced last year, assisting platforms like Facebook and Instagram with content ranking and recommendations.

Meta’s new generation AI chip will be produced by TSMC using the 5nm process. Meta reveals that Artemis offers triple the performance of the first-generation MTIA.

In October last year, Meta announced plans to invest USD 35 billion to establish infrastructure supporting AI, including data centers and hardware. CEO Mark Zuckerberg told investors, “In terms of investment priorities, AI will be our biggest investment area in 2024 for both engineering and compute resources.”

Meta’s proprietary AI chips are deployed in data centers to power AI applications. Meta has several ongoing projects aimed at expanding MTIA’s application scope, including supporting generative AI workloads.

The trend of tech giants developing their own AI chips is evident, with Meta joining competitors like Amazon, Microsoft, and Google in internal AI chip development to reduce reliance on NVIDIA. Google recently unveiled its latest data center AI chip, TPU v5p, on the 9th. Meanwhile, Intel is targeting NVIDIA’s H100 with its new AI chip, Gaudi 3.

Read more

• [News] NVIDIA’s H100 AI Chip No Longer Out of Reach, Inventory Pressure Reportedly Forces Customers to Resell

(Photo credit: Meta)

Amid NVIDIA’s leadership in the AI wave, demand for CoWoS (Chip-on-Wafer-on-Substrate) has tripled, driving TSMC to aggressively expand CoWoS capacity, with a corresponding surge in demand for System-in-Integrated-Circuit (SoIC) solutions.

According to a report from MoneyDJ citing industry sources, it has suggested that in addition to AMD, which has already implemented SoIC in production, Apple is conducting limited trial production. Furthermore, collaborations are underway with NVIDIA and Broadcom, indicating that SoIC is poised to become TSMC’s next advanced packaging solution following CoWoS.

TSMC’s SoIC is the industry’s first high-density 3D chip stacking technology, enabling heterogeneous integration of chips with different sizes, functionalities, and nodes using Chip on Wafer packaging. Currently, production takes place at the AP6 assembly and testing facility in Zhunan, Taiwan. It’s rumored that the planned advanced packaging facility in Chiayi, Taiwan will include not only two CoWoS plants but also an SoIC facility.

AMD is the first customer to adopt SoIC technology, with its latest MI300 chip using SoIC combined with CoWoS solution. Apple, TSMC’s primary customer, is reportedly interested in SoIC and plans to incorporate it with Hybrid molding technology for Mac products. Small-scale trials are currently underway, with mass production anticipated between 2025 and 2026. NVIDIA and Broadcom are also collaborating in this field.

As per the same report citing industry sources, the SoIC technology is still in its early stages, with monthly production capacity expected to reach around 2,000 wafers by the end of this year. There are prospects for this capacity to double this year and potentially exceed 10,000 wafers by 2027.

With support from major players like AMD, Apple, and NVIDIA, TSMC’s expansion in SoIC is viewed as confident, securing future orders for high-end chip manufacturing and advanced packaging.

Read more

• [News] TSMC Reportedly Sensing Increased Orders Again, CoWoS Production Capacity Surges
• [News] TSMC’s SoIC Demand Heats Up, Reports Suggest Significant Capacity Expansion

(Photo credit: TSMC)