Four major cloud service providers (CSPs) including Google, Microsoft, Amazon, and Meta, sequentially released their first-quarter financial performance for the year 2024 (January 2024 to March 2024) at the end of April.

Each company has achieved double-digit growth of the revenue, with increased capital expenditures continuing to emphasize AI as their main development focus. The market’s current focus remains on whether AI investment projects can successfully translate into revenue from the previous quarter to date.

TrendForce’s Insights:

1. Strong Financial Performance of Top Four CSPs Driven by AI and Cloud Businesses

Alphabet, the parent company of Google, reported stellar financial results for the first quarter of 2024. Bolstered by growth in search engine, YouTube, and cloud services, revenue surpassed USD 80 billion, marking a 57% increase in profit. The company also announced its first-ever dividend payout, further boosting its stock price as all metrics exceeded market expectations, pushing its market capitalization past USD 2 trillion for the first time.For Google, the current development strategy revolves around its in-house LLM Gemini layout, aimed at strengthening its cloud services, search interaction interfaces, and dedicated hardware development.

Microsoft’s financial performance is equally impressive. This quarter, its revenue reached USD 61.9 billion, marking a year-on-year increase of 17%. Among its business segments, the Intelligent Cloud sector saw the highest growth, with a 21% increase in revenue, totaling $26.7 billion. Notably, the Azure division experienced a remarkable 31% growth, with Microsoft attributing 7% of this growth to AI demand.

In other words, the impact of AI on its performance is even more pronounced than in the previous quarter, prompting Microsoft to focus its future strategies more on the anticipated benefits from Copilot, both in software and hardware.

This quarter, Amazon achieved a remarkable revenue milestone, surpassing USD 140 billion, representing a year-on-year increase of 17%, surpassing market expectations. Furthermore, its profit reached USD 10.4 billion, far exceeding the USD 3.2 billion profit recorded in the same period in 2023.

The double-digit growth in advertising business and AWS (Amazon Web Services) drove this performance, with the latter being particularly highlighted for its AI-related opportunities. AWS achieved a record-high operating profit margin of 37.6% this quarter, with annual revenue expected to exceed $100 billion, and short-term plans to invest USD 150 billion in expanding data centers.

On the other hand, Meta reported revenue of USD 36.46 billion this quarter, marking a significant year-on-year growth of 27%, the largest growth rate since 2021. Profit also doubled compared to the same period in 2023, reaching USD 12.37 billion.

Meta’s current strategy focuses on allocating resources to areas such as smart glasses and mixed reality (MR) in the short and medium term. The company continues to leverage AI to enhance the user value of the virtual world.

2. Increased Capital Expenditure to Develop AI is a Common Consensus, Yet Profitability Remains Under Market Scrutiny

Observing the financial reports of major cloud players, the increase in capital expenditure to solidify their commitment to AI development can be seen as a continuation of last quarter’s focus.

In the first quarter of 2024, Microsoft’s capital expenditure surged by nearly 80% compared to the same period in 2023, reaching USD 14 billion. Google expects its quarterly expenditure to remain above USD 12 billion. Similarly, Meta has raised its capital expenditure guidance for 2024 to the range of USD 35 to USD 40 billion.

Amazon, considering its USD 14 billion expenditure in the first quarter as the minimum for the year, anticipates a significant increase in capital expenditure over the next year, exceeding the USD 48.4 billion spent in 2023. However, how these increased investments in AI will translate into profitability remains a subject of market scrutiny.

While the major cloud players remain steadfast in their focus on AI, market expectations may have shifted. For instance, despite impressive financial reports last quarter, both Google and Microsoft saw declines in their stock prices, unlike the significant increases seen this time. This could partly be interpreted as an expectation of short- to medium-term AI investment returns from products and services like Gemini and Copilot.

In contrast, Meta, whose financial performance is similarly impressive to other cloud giants, experienced a post-earnings stock drop of over 15%. This may be attributed partly to its conservative financial outlook and partly to the less-than-ideal investment returns from its focused areas of virtual wearable devices and AI value-added services.

Due to Meta’s relatively limited user base compared to the other three CSPs in terms of commercial end-user applications, its AI development efforts, such as the practical Llama 3 and the value-added Meta AI virtual assistant for its products, have not yielded significant benefits. While Llama 3 is free and open-source, and Meta AI has limited shipment, they evidently do not justify the development costs.

Therefore, Meta still needs to expand its ecosystem to facilitate the promotion of its AI services, aiming to create a business model that can translate technology into tangible revenue streams.

For example, Meta recently opened up the operating system Horizon OS of its VR device Quest to brands like Lenovo and Asus, allowing them to produce their own branded VR/MR devices. The primary goal is to attract developers to enrich the content database and thereby promote industry development.

Read more

• [News] NVIDIA’s H100 Chip Heat Declines, CoWoS Capacity Eases
• [News] Microsoft Reportedly Stockpiling AI Chips to Boost Hardware Capability

In 2023, “generative AI” was undeniably the hottest term in the tech industry.

The launch of the generative application ChatGPT by OpenAI has sparked a frenzy in the market, prompting various tech giants to join the race.

As per a report from TechNews, currently, NVIDIA dominates the market by providing AI accelerators, but this has led to a shortage of their AI accelerators in the market. Even OpenAI intends to develop its own chips to avoid being constrained by tight supply chains.

On the other hand, due to restrictions arising from the US-China tech war, while NVIDIA has offered reduced versions of its products to Chinese clients, recent reports suggest that these reduced versions are not favored by Chinese customers.

Instead, Chinese firms are turning to Huawei for assistance or simultaneously developing their own chips, expected to keep pace with the continued advancement of large-scale language models.

In the current wave of AI development, NVIDIA undoubtedly stands as the frontrunner in AI computing power. Its A100/H100 series chips have secured orders from top clients worldwide in the AI market.

As per analyst Stacy Rasgon from the Wall Street investment bank Bernstein Research, the cost of each query using ChatGPT is approximately USD 0.04. If ChatGPT queries were to scale to one-tenth of Google’s search volume, the initial deployment would require approximately USD 48.1 billion worth of GPUs for computation, with an annual requirement of about USD 16 billion worth of chips to sustain operations, along with a similar amount for related chips to execute tasks.

Therefore, whether to reduce costs, decrease overreliance on NVIDIA, or even enhance bargaining power further, global tech giants have initiated plans to develop their own AI accelerators.

Per reports by technology media The Information, citing industry sources, six global tech giants, including Microsoft, OpenAI, Tesla, Google, Amazon, and Meta, are all investing in developing their own AI accelerator chips. These companies are expected to compete with NVIDIA’s flagship H100 AI accelerator chips.

Progress of Global Companies’ In-house Chip Development

• Microsoft

Rumors surrounding Microsoft’s in-house AI chip development have never ceased.

At the annual Microsoft Ignite 2023 conference, the company finally unveiled the Azure Maia 100 AI chip for data centers and the Azure Cobalt 100 cloud computing processor. In fact, rumors of Microsoft developing an AI-specific chip have been circulating since 2019, aimed at powering large language models.

The Azure Maia 100, introduced at the conference, is an AI accelerator chip designed for tasks such as running OpenAI models, ChatGPT, Bing, GitHub Copilot, and other AI workloads.

According to Microsoft, the Azure Maia 100 is the first-generation product in the series, manufactured using a 5-nanometer process. The Azure Cobalt is an Arm-based cloud computing processor equipped with 128 computing cores, offering a 40% performance improvement compared to several generations of Azure Arm chips. It provides support for services such as Microsoft Teams and Azure SQL. Both chips are produced by TSMC, and Microsoft is already designing the second generation.

• Open AI

OpenAI is also exploring the production of in-house AI accelerator chips and has begun evaluating potential acquisition targets. According to earlier reports from Reuters citing industry sources, OpenAI has been discussing various solutions to address the shortage of AI chips since at least 2022.

Although OpenAI has not made a final decision, options to address the shortage of AI chips include developing their own AI chips or further collaborating with chip manufacturers like NVIDIA.

OpenAI has not provided an official comment on this matter at the moment.

• Tesla

Electric car manufacturer Tesla is also actively involved in the development of AI accelerator chips. Tesla primarily focuses on the demand for autonomous driving and has introduced two AI chips to date: the Full Self-Driving (FSD) chip and the Dojo D1 chip.

The FSD chip is used in Tesla vehicles’ autonomous driving systems, while the Dojo D1 chip is employed in Tesla’s supercomputers. It serves as a general-purpose CPU, constructing AI training chips to power the Dojo system.

• Google

Google began secretly developing a chip focused on AI machine learning algorithms as early as 2013 and deployed it in its internal cloud computing data centers to replace NVIDIA’s GPUs.

The custom chip, called the Tensor Processing Unit (TPU), was unveiled in 2016. It is designed to execute large-scale matrix operations for deep learning models used in natural language processing, computer vision, and recommendation systems.

In fact, Google had already constructed the TPU v4 AI chip in its data centers by 2020. However, it wasn’t until April 2023 that technical details of the chip were publicly disclosed.

• Amazon

As for Amazon Web Services (AWS), the cloud computing service provider under Amazon, it has been a pioneer in developing its own chips since the introduction of the Nitro1 chip in 2013. AWS has since developed three product lines of in-house chips, including network chips, server chips, and AI machine learning chips.

Among them, AWS’s lineup of self-developed AI chips includes the inference chip Inferentia and the training chip Trainium.

On the other hand, AWS unveiled the Inferentia 2 (Inf2) in early 2023, specifically designed for artificial intelligence. It triples computational performance while increasing accelerator total memory by a quarter.

It supports distributed inference through direct ultra-high-speed connections between chips and can handle up to 175 billion parameters, making it the most powerful in-house manufacturer in today’s AI chip market.

• Meta

Meanwhile, Meta, until 2022, continued using CPUs and custom-designed chipsets tailored for accelerating AI algorithms to execute its AI tasks.

However, due to the inefficiency of CPUs compared to GPUs in executing AI tasks, Meta scrapped its plans for a large-scale rollout of custom-designed chips in 2022. Instead, it opted to purchase NVIDIA GPUs worth billions of dollars.

Still, amidst the surge of other major players developing in-house AI accelerator chips, Meta has also ventured into internal chip development.

On May 19, 2023, Meta further unveiled its AI training and inference chip project. The chip boasts a power consumption of only 25 watts, which is 1/20th of the power consumption of comparable products from NVIDIA. It utilizes the RISC-V open-source architecture. According to market reports, the chip will also be produced using TSMC’s 7-nanometer manufacturing process.

China’s Progress on In-House Chip Development

China’s journey in developing in-house chips presents a different picture. In October last year, the United States expanded its ban on selling AI chips to China.

Although NVIDIA promptly tailored new chips for the Chinese market to comply with US export regulations, recent reports suggest that major Chinese cloud computing clients such as Alibaba and Tencent are less inclined to purchase the downgraded H20 chips. Instead, they have begun shifting their orders to domestic suppliers, including Huawei.

This shift in strategy indicates a growing reliance on domestically developed chips from Chinese companies by transferring some orders for advanced semiconductors to China.

TrendForce indicates that currently about 80% of high-end AI chips purchased by Chinese cloud operators are from NVIDIA, but this figure may decrease to 50% to 60% over the next five years.

If the United States continues to strengthen chip controls in the future, it could potentially exert additional pressure on NVIDIA’s sales in China.

Read more

• [News] NVIDIA’s Exclusive Chips for China Now Reported to be Available for Pre-Order, Priced Similar to Huawei Products

(Photo credit: NVIDIA)

On the 28th, Amazon unveiled two AWS-designed chips, Graviton4, a CPU propelling its AWS cloud services, and the second-gen AI chip Trainium2, tailored for large language models. Both chips boast substantial performance upgrades. With a positive market outlook, Amazon is intensifying its competition with Microsoft and Google for dominance in the AI cloud market. The demand for in-house chips is surging, leading to increased orders for key players like the wafer foundry TSMC and the silicon design and production services company ALCHIP, reported by UDN News.

According to reports, Amazon AWS CEO Adam Selipsky presented the fourth AWS-Designed custom CPU chip, Graviton4, at the AWS re:Invent 2023 in Las Vegas. It claims a 30% improvement in computing performance compared to the current Graviton3, with a 75% increase in memory bandwidth. Computers equipped with this processor are slated to go live in the coming months.

Trainium2, the second-gen chip for AI system training, boasts a computing speed three times faster than its predecessor and doubled energy efficiency. Selipsky announced that AWS will commence offering this new training chip next year.

AWS is accelerating the development of chips, maintaining its lead over Microsoft Azure and Google Cloud platforms. Amazon reports that over 50,000 AWS customers are currently utilizing Graviton chips.

Notably, Amazon’s in-house chip development heavily relies on the Taiwan supply chain, TSMC and ALchip. To produce Amazon’s chips, Alchip primarily provides application-specific integrated circuit (ASIC) design services, and TSMC manufactures with advanced processes.

TSMC consistently refrains from commenting on products for individual customers. Analysts estimate that TSMC has recently indirectly secured numerous orders from Cloud Service Providers (CSPs), mainly through ASIC design service providers assisting CSP giants in launching new in-house AI chips. This is expected to significantly contribute to TSMC’s high utilization for the 5nm family.

In recent years, TSMC has introduced successive technologies such as N4, N4P, N4X, and N5A to strengthen its 5nm family. The N4P, announced at 2023 Technology Symposium, is projected to drive increased demand from 2024 onwards. The expected uptick in demand is mainly attributed to AI, network, and automotive products.

(Image: Amazon)

Microsoft announced the in-house AI chip, Azure Maia 100, at the Ignite developer conference in Seattle on November 15, 2023. This chip is designed to handle OpenAI models, Bing, GitHub Copilot, ChatGPT, and other AI services. Support for Copilot, Azure OpenAI is expected to commence in early 2024.

TrendForce’s Insights:

1. Speculating on the Emphasis of Maia 100 on Inference, Microsoft’s Robust Ecosystem Advantage is Poised to Emerge Gradually

Microsoft has not disclosed detailed specifications for Azure Maia 100. Currently, it is known that the chip will be manufactured using TSMC’s 5nm process, featuring 105 billion transistors and supporting at least INT8 and INT4 precision formats. While Microsoft has indicated that the chip will be used for both training and inference, the computational formats it supports suggest a focus on inference applications.

This emphasis is driven by its incorporation of the less common INT4 low-precision computational format in comparison to other CSP manufacturers’ AI ASICs. Additionally, the lower precision contributes to reduced power consumption, shortening inference times, enhancing efficiency. However, the drawback lies in the sacrifice of accuracy.

Microsoft initiated its in-house AI chip project, “Athena,” in 2019. Developed in collaboration with OpenAI. Azure Maia 100, like other CSP manufacturers, aims to reduce costs and decrease dependency on NVIDIA. Despite Microsoft entering the field of proprietary AI chips later than its primary competitors, its formidable ecosystem is expected to gradually demonstrate a competitive advantage in this regard.

2. U.S. CSP Manufacturers Unveil In-House AI Chips, Meta Exclusively Adopts RISC-V Architecture

Google led the way with its first in-house AI chip, TPU v1, introduced as early as 2016, and has since iterated to the fifth generation with TPU v5e. Amazon followed suit in 2018 with Inferentia for inference, introduced Trainium for training in 2020, and launched the second generation, Inferentia2, in 2023, with Trainium2 expected in 2024.

Meta plans to debut its inaugural in-house AI chip, MTIA v1, in 2025. Given the releases from major competitors, Meta has expedited its timeline and is set to unveil the second-generation in-house AI chip, MTIA v2, in 2026.

Unlike other CSP manufacturers, both MTIA v1 and MTIA v2 adopt the RISC-V architecture, while other CSP manufacturers opt for the ARM architecture. RISC-V is a fully open-source architecture, requiring no instruction set licensing fees. The number of instructions (approximately 200) in RISC-V is lower than ARM (approximately 1,000).

This choice allows chips utilizing the RISC-V architecture to achieve lower power consumption. However, the RISC-V ecosystem is currently less mature, resulting in fewer manufacturers adopting it. Nevertheless, with the growing trend in data centers towards energy efficiency, it is anticipated that more companies will start incorporating RISC-V architecture into their in-house AI chips in the future.

3. The Battle of AI Chips Ultimately Relies on Ecosystems, Microsoft Poised for Competitive Edge

The competition among AI chips will ultimately hinge on the competition of ecosystems. Since 2006, NVIDIA has introduced the CUDA architecture, nearly ubiquitous in educational institutions. Thus, almost all AI engineers encounter CUDA during their academic tenure.

In 2017, NVIDIA further solidified its ecosystem by launching the RAPIDS AI acceleration integration solution and the GPU Cloud service platform. Notably, over 70% of NVIDIA’s workforce comprises software engineers, emphasizing its status as a software company. The performance of NVIDIA’s AI chips can be further enhanced through software innovations.

On the contrary, Microsoft possess a robust ecosystem like Windows. The recent Intel Arc GPU A770 showcased a 1.7x performance improvement in AI-driven Stable Diffusion on Microsoft Olive, this demonstrates that, similar to NVIDIA, Microsoft has the capability to enhance GPU performance through software.

Consequently, Microsoft’s in-house AI chips are poised to achieve superior performance in software collaboration compared to other CSP manufacturers, providing Microsoft with a competitive advantage in the AI competition.

Read more

• [News] Microsoft First In-House AI Chip “Maia” Produced by TSMC’s 5nm

According to a report by Taiwan’s Economic Daily, TSMC’s CoWoS advanced packaging capacity is running at full throttle. As they actively expand their production capabilities, there are reports of major customers like NVIDIA increasing their orders for AI chips. Additionally, industry giants like AMD and Amazon have rushed in with urgent orders.

In response to this urgent situation, TSMC is actively seeking equipment suppliers to expand its CoWoS machine procurement. Beyond TSMC’s existing production expansion goals, the company is further increasing its orders for equipment by an additional 30%, highlighting the ongoing fervor in the AI market.

It is reported that TSMC has sought assistance from equipment manufacturers such as Scientech, Allring, Grand Process Technology, E&R Engineering, and GP Group for this endeavor. They plan to complete the delivery and installation of the equipment by the first half of the coming year. The related equipment manufacturers are experiencing a surge in activity.

Industry sources reveal that TSMC’s CoWoS advanced packaging monthly production capacity is currently around 12,000 units. With their previous expansion efforts, they aimed to gradually increase this to 15,000 to 20,000 units per month. Now, with the addition of more equipment, they are looking at the possibility of reaching capacities of over 25,000 units per month, potentially even approaching 30,000 units. This substantial increase in production capacity positions TSMC to handle a significantly larger volume of AI-related orders.

Equipment providers have pointed out that NVIDIA is currently TSMC’s largest customer for CoWoS advanced packaging, accounting for 60% of the production capacity. Recently, in response to robust demand in AI computing, NVIDIA has increased its orders. Additionally, urgent orders from other customers such as AMD, Amazon, and Broadcom have started to pour in.

(Photo credit: TSMC)