AMD has long aspired to gain more favor for its AI chips, aiming to break into Nvidia’s stronghold in the AI chip market. Key players like Meta, OpenAI, and Microsoft, who are major buyers of AI chips, also desire a diversified market with multiple AI chip suppliers to avoid vendor lock-in issues and reduce costs.

With AMD’s latest AI chip, Instinct MI300X slated for significant shipments in early 2024, these three major AI chip buyers have publicly announced their plans to place orders as they consider AMD’s solution a more cost-effective alternative.

At the AMD “Advancing AI” event on December 6th, Meta, OpenAI, Microsoft, and Oracle declared their preference for AMD’s latest AI chip, Instinct MI300X. This marks a groundbreaking move by AI tech giants actively seeking alternatives to Nvidia’s expensive GPUs.

For applications like OpenAI’s ChatGPT, Nvidia GPUs have played a crucial role. However, if the AMD MI300X can provide a significant cost advantage, it has the potential to impact Nvidia’s sales performance and challenge its market dominance in AI chips.

AMD’s Three Major Challenges

AMD grapples with three major challenges: convincing enterprises to consider substitutions, addressing industry standards compared to Nvidia’s CUDA software, and determining competitive GPU pricing. Lisa Su, AMD’s CEO, highlighted at the event that the new MI300X architecture features 192GB of high-performance HBM3, delivering not only faster data transfer but also meeting the demands of larger AI models. Su emphasized that such a notable performance boost translates directly into an enhanced user experience, enabling quicker responses to complex user queries.

However, AMD is currently facing critical challenges. Companies that heavily rely on Nvidia may hesitate to invest their time and resources in an alternative GPU supplier like AMD. Su believes that there is an opportunity to make efforts in persuading these AI tech giants to adopt AMD GPUs.

Another pivotal concern is that Nvidia has established its CUDA software as the industry standard, resulting in a highly loyal customer base. In response, AMD has made improvements to its ROCm software suite to effectively compete in this space. Lastly, pricing is a crucial issue, as AMD did not disclose the price of the MI300X during the event. Convincing customers to choose AMD over Nvidia, whose chips are priced around USD 40,000 each, will require substantial cost advantages in both the purchase and operation of AMD’s offerings.

The Overall Size of the AI GPU Market is Expected to Reach USD 400 Billion by 2027

AMD has already secured agreements with companies eager for high-performance GPUs to use MI300X. Meta plans to leverage MI300X GPUs for AI inference tasks like AI graphics, image editing, and AI assistants. On the other hands, Microsoft’s CTO, Kevin Scott, announced that the company will provide access to MI300X through Azure web service.

Additionally, OpenAI has decided to have its GPU programming language Triton, a dedication to machine learning algorithm development, support AMD MI300X. Oracle Cloud Infrastructure (OCI) intends to introduce bare-metal instances based on AMD MI300X GPUs in its high-performance accelerated computing instances for AI.

AMD anticipates that the annual revenue from its GPUs for data centers will reach USD 2 billion by 2024. This projected figure is substantially lower than Nvidia’s most recent quarterly sales related to the data center business (i.e., over USD 14 billion, including sales unrelated to GPUs). AMD emphasizes that with the rising demand for high-end AI chips, the AI GPU market’s overall size is expected to reach USD 400 billion by 2027. This strategic focus on AI GPU products underscores AMD’s optimism about capturing a significant market share. Lisa Su is confident that AMD is poised for success in this endeavor.

(Image: AMD)

According to the South Korean media The Korea Economic Daily’s report, Samsung Electronics has established a new business unit dedicated to developing next-generation chip processing technology. The aim is to secure a leading position in the field of AI chips and foundry services.

The report indicates that the recently formed research team at Samsung will be led by Hyun Sang-jin, who was promoted to the position of general manager on November 29. He has been assigned the responsibility of ensuring a competitive advantage against competitors like TSMC in the technology landscape.

The team will be placed under Samsung’s chip research center within its Device Solutions (DS) division, which oversees its semiconductor business, as mentioned in the report.

Reportedly, insiders claim that Samsung aims for the latest technology developed by the team to lead the industry for the next decade or two, similar to the gate-all-around (GAA) transistor technology introduced by Samsung last year.

Samsung has previously stated that compared to the previous generation process, the 3-nanometer GAA process can deliver a 30% improvement in performance, a 50% reduction in power consumption, and a 45% reduction in chip size. In the report, Samsung also claimed that it is more energy-efficient compared to FinFET technology, which is utilized by the TSMC’s 3-nanometer process.

Read more

• [News] IC Design Companies Seek Advanced Process Second Source, Overview of Competition Between TSMC and Samsung

(Photo credit: Samsung)

MediaTek announced a collaboration with Meta to develop its next-generation smart glasses chip. Since Meta has previously used Qualcomm chips for its two generations of smart glasses products, it is speculated that Meta’s expansion of chip suppliers is aimed at maintaining supply chain flexibility and reducing costs. MediaTek, in turn, is poised to leverage smart glasses to tap into opportunities within Meta’s VR/AR devices.

 TrendForce’s Insights:

1. Meta Expands Chip Collaboration Suppliers, Maintaining Product Development Flexibility and Potential Cost Reduction

In mid-November 2023, MediaTek hosted the overseas summit, Mediatek Executive Summit 2023, where it announced a collaboration with Meta to develop the next-generation smart glasses chip.

Meta’s first smart glasses, a collaborative creation with Ray-Ban in 2021, differ from the Quest series as they are not high-end VR devices but rather feature a simpler design, focusing on additional functionalities like music playback and phone calls.

In the fall of 2023, Meta introduced a successor product with significant improvements in camera resolution, video quality, microphones, and internal storage. This new device is designed to simplify the recording and live streaming process by integrating with Meta’s social platform. Additionally, the new product aligns with the trend of generative AI and incorporates Meta’s AI voice assistant based on Llama2 LLM.

Notably, the market has shown keen interest and discussion regarding MediaTek’s announcement on the collaboration with Meta, given that Meta’s previous two generations of smart glasses used Qualcomm chips, specifically the Qualcomm Snapdragon Wear 4100 for the older version and the AR1 Gen1 for the new version.

Analysis of Meta’s Motivation: Meta’s decision to collaborate with MediaTek may be driven by considerations of risk diversification among suppliers and overall cost reduction.

Firstly, Meta has been investing in the development of in-house chips in recent years to ensure flexibility in product development. Examples include the MTIA chip, disclosed in mid-2023, designed for processing inference-related tasks, and the MSVP, the first in-house ASIC chip for video transcoding, which is expected to be used in VR and AR devices.

Given Meta’s previous attempts, including collaboration with Samsung, to independently develop chips and move towards chip autonomy, the partnership with MediaTek can be seen as a risk mitigation strategy against vendor lock-in.

Secondly, considering that smart glasses, unlike the high-priced Quest series, are currently priced at USD 299 for both models, MediaTek’s competitive pricing may also be a significant factor in Meta’s decision to collaborate with them.

2. MediaTek Eyes VR and AR Device Market Opportunities Through Smart Glasses Collaboration with Meta

From MediaTek’s perspective, their focus extends beyond smart glasses to the vast business opportunities presented by Meta’s VR and AR devices. In reality, examining Meta’s smart glasses alone reveals estimated shipments of around 300,000 pairs for the older model. Even with the new model and the anticipated successor expected to launch in 2025, there is currently no clear indication of significant market momentum.

In practical terms, this collaboration with Meta might not contribute substantially to MediaTek’s revenue. The crucial aspect of MediaTek’s collaboration with Meta lies in strategically positioning itself in Meta’s smart headwear supply chain, challenging the dominance of the original chip supplier, Qualcomm.

Looking at global VR device shipments, Meta is projected to hold over 70% market share in 2023 and 2024. There are also reports of an updated version of the Quest device expected to be available in China in late 2024. If MediaTek can expand its collaboration with Meta further, coupled with the gradual increase in the penetration rate of VR and AR devices, significant business opportunities still lie ahead.

From an overall perspective of the VR and AR industry, the current design of headwear devices no longer resembles the early models that required external computing cores due to considerations of cost, power, and heat.

The prevalent mainstream designs are now standalone devices. Given that these devices not only execute the primary application functions but also handle and consolidate a substantial amount of data from sensors to support functions like object tracking and image recognition, VR and AR devices require high-performance chips or embedded auxiliary SoCs. This market demand and profit potential are compelling enough to attract chip manufacturers, especially in the face of the gradual decline in momentum in the consumer electronics market, such as smartphones.

The VR and AR market still holds development potential, making it a strategic entry point for manufacturers. This insight is evident in MediaTek’s motivation, continuing its market cultivation efforts after developing the first VR chip for Sony PS VR2 in 2022 and collaborating with Meta.

According to TechNews’ report, during a recent financial conference, Samsung revealed its plans to diversify its sales structure by expanding its clientele in the fields of artificial intelligence semiconductors and automotive, moving away from its previous heavy reliance on the mobile sector.

As of 2023, it is understood that Samsung’s foundry sales distribution includes 54% from mobile, 19% from high-performance computing, and 11% from automotive.

According to a report from Wccftech, senior executives at Samsung have indicated that major players such as super-scale data centers, automotive original equipment manufacturers (OEMs), and other clients have been in contact with Samsung, considering the adoption of Samsung’s foundry services to manufacture their designed chips.

This includes the in-development 4-nanometer artificial intelligence accelerator and the 5-nanometer chips for the top-ranked electric vehicle company. Currently, Samsung is gearing up with its advanced packaging solution called SAINT (Samsung Advanced Interconnection Technology), aiming to compete with TSMC’s advanced packaging, CoWoS. Based on information disclosed by Samsung, there might be a collaboration with AMD in the field of artificial intelligence, involving the manufacturing of certain chips.

In fact, recent rumors suggest that Samsung has already reached an agreement with AMD to provide HBM3 and packaging technology for the upcoming Instinct MI300 series. Additionally, AMD might adopt a dual-sourcing strategy for the Zen 5 series architecture, choosing TSMC’s 3-nanometer process and Samsung’s 4-nanometer process technology for manufacturing the next-generation chips.

According to sources, besides the artificial intelligence domain, Samsung is likely to have received orders from the electric vehicle giant Tesla. The speculation points towards the possibility of fulfilling orders for Tesla’s next-generation HW 5.0 chip, designed for fully autonomous driving applications.

Read more

• [News] Samsung Reportedly Lands a 4nm Mega Order – Why is AMD Switching to “Dual Foundry Mode” for Its Next-Gen Chips?

(Photo credit: Samsung)

In the global landscape of self-developed chips, the industry has predominantly embraced the Arm architecture for IC design. However, Meta’s decision to employ the RISC-V architecture in its self-developed AI chip has become a topic of widespread discussion. It is said the growing preference for RISC-V is attributed to three key advantages including low power consumption, high openness, and relatively lower development costs, according to reports from UDN News.

Noted that Meta exclusively deploys its in-house AI chip, “MTIA,” within its data centers to expedite AI computation and inference. In this highly tailored setting, this choice ensures not only robust computational capabilities but also the potential for low power consumption, with an anticipated power usage of under 25W per RISC-V core. By strategically combining the RISC-V architecture with GPU accelerators or Arm architecture, Meta aims to achieve an overall reduction in power consumption while boosting computing power simultaneously.

Meta’s confirmation of adopting RISC-V architecture form Andes Technology Corporation, a CPU IP and Platform IP supplier from Taiwan, for AI chip development underscores RISC-V’s capability to support high-speed computational tasks and its suitability for integration into advanced manufacturing processes. This move positions RISC-V architecture to potentially make significant inroads into the AI computing market,  and stands as the third computing architecture opportunity, joining the ranks of x86 and Arm architectures.

Regarding the development potential of different chip architectures in the AI chip market, TrendForce points out that in the current overall AI market, GPUs (such as NVIDIA, AMD, etc.) still dominate, followed by Arm architecture. This includes major data centers, with active investments from NVIDIA, CSPs, and others in the Arm architecture field. RISC, on the other hand, represents another niche market, targeting the open-source AI market or enterprise niche applications.
(Image: Meta)