NVIDIA’s latest financial report for FY2Q24 reveals that its data center business reached US$10.32 billion—a QoQ growth of 141% and YoY increase of 171%. The company remains optimistic about its future growth. TrendForce believes that the primary driver behind NVIDIA’s robust revenue growth stems from its data center’s AI server-related solutions. Key products include AI-accelerated GPUs and AI server HGX reference architecture, which serve as the foundational AI infrastructure for large data centers.

TrendForce further anticipates that NVIDIA will integrate its software and hardware resources. Utilizing a refined approach, NVIDIA will align its high-end, mid-tier, and entry-level GPU AI accelerator chips with various ODMs and OEMs, establishing a collaborative system certification model. Beyond accelerating the deployment of CSP cloud AI server infrastructures, NVIDIA is also partnering with entities like VMware on solutions including the Private AI Foundation. This strategy extends NVIDIA’s reach into the edge enterprise AI server market, underpinning steady growth in its data center business for the next two years.

NVIDIA’s data center business surpasses 76% market share due to strong demand for cloud AI

In recent years, NVIDIA has been actively expanding its data center business. In FY4Q22, data center revenue accounted for approximately 42.7%, trailing its gaming segment by about 2 percentage points. However, by FY1Q23, data center business surpassed gaming—accounting for over 45% of revenue. Starting in 2023, with major CSPs heavily investing in ChatBOTS and various AI services for public cloud infrastructures, NVIDIA reaped significant benefits. By FY2Q24, data center revenue share skyrocketed to over 76%.

NVIDIA targets both Cloud and Edge Data Center AI markets

TrendForce observes and forecasts a shift in NVIDIA’s approach to high-end GPU products in 2H23. While the company has primarily focused on top-tier AI servers equipped with the A100 and H100, given positive market demand, NVIDIA is likely to prioritize the higher-priced H100 to effectively boost its data-center-related revenue growth.

NVIDIA is currently emphasizing the L40s as their flagship product for mid-tier GPUs, meaning several strategic implications: Firstly, the high-end H100 series is constrained by the limited production capacity of current CoWoS and HBM technologies. In contrast, the L40s primarily utilizes GDDR memory. Without the need for CoWos packaging, it can be rapidly introduced to the mid-tier AI server market, filling the gap left by the A100 PCle interface in meeting the needs of enterprise customers.

Secondly, the L40s also target enterprise customers who don’t require large parameter models like ChatGPT. Instead, it focuses on more compact AI training applications in various specialized fields, with parameter counts ranging from tens of billions to under a hundred billion. They can also address edge AI inference or image analysis tasks. Additionally, in light of potential geopolitical issues that might disrupt the supply of the high-end GPU H series for Chinese customers, the L40s can serve as an alternative. As for lower-tier GPUs, NVIDIA highlights the L4 or T4 series, which are designed for real-time AI inference or image analysis in edge AI servers. These GPUs underscore affordability while maintaining a high-cost-performance ratio.

HGX and MGX AI server reference architectures are set to be NVIDIA’s main weapons for AI solutions in 2H23

TrendForce notes that recently, NVIDIA has not only refined its product positioning for its core AI chip GPU but has also actively promoted its HGX and MGX solutions. Although this approach isn’t new in the server industry, NVIDIA has the opportunity to solidify its leading position with this strategy. The key is NVIDIA’s absolute leadership stemming from its extensive integration of its GPU and CUDA platform—establishing a comprehensive AI ecosystem. As a result, NVIDIA has considerable negotiating power with existing server supply chains. Consequently, ODMs like Inventec, Quanta, FII, Wistron, and Wiwynn, as well as brands such as Dell, Supermicro, and Gigabyte, are encouraged to follow NVIDIA’s HGX or MGX reference designs. However, they must undergo NVIDIA’s hardware and software certification process for these AI server reference architectures. Leveraging this, NVIDIA can bundle and offer integrated solutions like its Arm CPU Grace, NPU, and AI Cloud Foundation.

It’s worth noting that for ODMs or OEMs, given that NVIDIA is expected to make significant achievements in the AI server market for CSPs from 2023 to 2024, there will likely be a boost in overall shipment volume and revenue growth of AI servers. However, with NVIDIA’s strategic introduction of standardized AI server architectures like HGX or MGX, the core product architecture for AI servers among ODMs and others will become more homogenized. This will intensify the competition among them as they vie for orders from CSPs. Furthermore, it’s been observed that large CSPs such as Google and AWS are leaning toward adopting in-house ASIC AI accelerator chips in the future, meaning there’s a potential threat to a portion of NVIDIA’s GPU market. This is likely one of the reasons NVIDIA continues to roll out GPUs with varied positioning and comprehensive solutions. They aim to further expand their AI business aggressively to Tier-2 data centers (like CoreWeave) and edge enterprise clients.

According to the Korea Economic Daily. Samsung Electronics’ HBM3 and packaging services have passed AMD’s quality tests. The upcoming Instinct MI300 series AI chips from AMD are planned to incorporate Samsung’s HBM3 and packaging services. These chips, which combine central processing units (CPUs), graphics processing units (GPUs), and HBM3, are expected to be released in the fourth quarter of this year.

Samsung is noted as the sole provider capable of offering advanced packaging solutions and HBM products simultaneously. Originally considering TSMC’s advanced packaging services, AMD had to alter its plans due to capacity constraints.

The surge in demand for high-performance GPUs within the AI landscape benefits not only GPU manufacturers like NVIDIA and AMD, but also propels the development of HBM and advanced packaging.

In the backdrop of the AI trend, AIGC model training and inference require the deployment of AI servers. These servers typically require mid-to-high-end GPUs, with HBM penetration nearing 100% among these GPUs.

Presently, Samsung, SK Hynix, and Micron are the primary HBM manufacturers. According to the latest research by TrendForce, driven by the expansion efforts of these original manufacturers, the estimated annual growth rate of HBM supply in 2024 is projected to reach 105%.

In terms of competitive dynamics, SK Hynix leads with its HBM3 products, serving as the primary supplier for NVIDIA’s Server GPUs. Samsung, on the other hand, focuses on fulfilling orders from other cloud service providers. With added orders from customers, the gap in market share between Samsung and SK Hynix is expected to narrow significantly this year. The estimated HBM market share for both companies is about 95% for 2023 to 2024. However, variations in customer composition might lead to sequential variations in bit shipments.

In the realm of advanced packaging capacity, TSMC’s CoWoS packaging technology dominates as the main choice for AI server chip suppliers. Amidst strong demand for high-end AI chips and HBM, TrendForce estimates that TSMC’s CoWoS monthly capacity could reach 12K by the end of 2023.

With strong demand driven by NVIDIA’s A100 and H100 AI Server requirements, demand for CoWoS capacity is expected to rise by nearly 50% compared to the beginning of the year. Coupled with the growth in high-end AI chip demand from companies like AMD and Google, the latter half of the year could experience tighter CoWoS capacity. This robust demand is expected to continue into 2024, potentially leading to a 30-40% increase in advanced packaging capacity, contingent on equipment readiness.

(Photo credit: Samsung)

According to Taiwan’s Economic Daily, the consumer market is experiencing starkly low demand, causing IC design firms primarily relying on mature processes, such as those in driver ICs, power management ICs, CMOS image sensors (CIS), and microcontrollers (MCUs), to adopt a notably cautious approach in placing orders. Some manufacturers are hesitating to place orders due to persistently high inventory levels.

The industry consensus is that IC design companies are expected to increase their orders in mature processes, with the earliest effects possibly emerging by 2024, implying that the mature process market conditions might not improve significantly until the end of this year.

The consumer market entered a period of economic downturn in the latter half of last year, which in turn affected industries such as PCs, smartphones, and networking. This not only led to a surge in inventory levels for IC design firms but also significantly curtailed the momentum for chip tape-out. Looking ahead to the second half of this year, while inventory levels across various sectors have largely returned to normal, chip tape-out for Q3 have notably declined compared to Q2.

In particular, demand for high-speed I/O in the PC sector and Board Management Controller for data centers remains notably weak. The supply chain indicates that PC demand for the second quarter, driven by advanced stocking, has dampened the typical peak season effect for the latter half of the year. This trend is evident across desktop PCs, laptops, and Chromebooks.”

As for the smartphone sector, after various research institutions revised down this year’s smartphone market size, the supply chain’s chip tape-out momentum has cooled down significantly. Only Qualcomm has increased its tape-out momentum to semiconductor foundries in the first half of the year, while MediaTek continues to adhere to a conservative strategy as of now.

(Photo credit: SMIC)

According to a report by Taiwan’s Economic Daily, the revival of the smartphone market has fallen short of expectations. Industry sources have indicated that in order to stimulate customer demand and expedite inventory clearance, Qualcomm has recently initiated a price war, targeting mid- to- low-range 5G smartphone chips. The price reduction is substantial, ranging from 10% to 20%. It is anticipated that Qualcomm’s price reduction strategy will extend into the fourth quarter.

The consumer electronics market began to slump in the fourth quarter of last year. Downstream inventory levels began to visibly dissipate in the first half of this year, gradually returning to normal. There was optimism in the market that the Chinese smartphone market would improve in the second half of this year, and there were even reports of a slight resurgence in Qualcomm’s chip shipments during the second quarter.

However, even after China’s 618 shopping festival, the downturn in the consumer electronics market has not shown significant improvement. This has led to Qualcomm’s inventory levels rising to nearly two quarters’ worth.

With low order visibility and high inventory, the supply chain has reported that Qualcomm has recently decided to initiate a price war, primarily focusing on the mid- to low-range market segment. If the pace of inventory clearance falls short of expectations, there is a possibility of further intensifying the price reduction efforts.

Industry analysts suggest that Qualcomm’s extensive price cuts underscore the challenging situation in the mid- to low-range 5G smartphone market, where demand has been lackluster.

(Photo credit: Qualcomm)

According to the latest report from TrendForce, the primary factors influencing the global market share of notebook CPUs in 2024 can be categorized into “Architectural Design” and “Economic Factors.”

“Architectural Design” as a long-term factor affecting market share:

(1) Both AMD (AMD 3D V-Cache) and Intel (Intel Foveros Direct) may potentially integrate 3D packaging technology into notebook computers in the future.

(2) Apple’s M-series processors, using the Arm core architecture, as well as Intel processors, have adopted a big/little core hybrid design. AMD might also introduce this in the Ryzen 8000 series.

(3) Despite further advancements in processor technology by 2024, the notebook computer market remains highly sensitive to the cost for IT equipment.

“Economic Factors” as more immediate influencers of market share:

(1) Until 2024, a return to lower interest rates in the global economic environment could favor corporate expansion of capital expenditure. This could result in increased procurement of business-oriented notebook models, potentially allowing Intel to further expand its CPU market share beyond 70% in the business sector.

(2) Concerns about economic prospects among global citizens until 2024 could have significant negative implications for the consumer notebook computer market. With the restart of physical economic activities, the demand for consumer-oriented notebook models has declined from the high levels seen during the pandemic. Consequently, the consumer market demand outlook for 2024 remains uncertain. For AMD, which relies more on consumer market demand, changes in market share may be harder to predict compared to Intel.

In the post-pandemic era, AMD, Arm/Apple, and Intel are pursuing distinct technological competition strategies to capture market share in the personal computing market.

AMD:

(1) The Socket AM5 platform is poised to aid AMD CPUs in achieving substantial performance and efficiency gains.

(2) The AMD Ryzen 7040 incorporates an artificial intelligence engine to emphasize AI computing performance’s importance in the thin and light notebook market.

Arm/Apple:

(1) The M2 Ultra processor heralds Apple’s complete transition of personal computing products to the Arm core. Apple Mac computer products will no longer be sold with Intel processor.

(2) The Apple M-series processors, built on the Arm core architecture, facilitate a “fanless design” to maintain MacBook’s slim profile. This feature highlights its irreplaceable positioning in the portable notebook computer market, emphasizing portability.

Intel:

(1) With the waning trend of the “hybrid work mode,” Intel is optimistic about diversified development in the post-pandemic era for desktop computer products. This includes microcomputers, micro workstations, and general workstations. Due to the characteristic of continuous operation for 24 hours, desktop computers still possess unique attributes that cannot be replaced by notebook computers.

(Photo credit: Intel)