The market was originally concerned that NVIDIA might face a demand lull during the transition from its Hopper series GPUs to the Blackwell series. However, the company executives clearly stated during the latest financial report release that this is not the case.

According to reports from MarketWatch and CNBC, NVIDIA CFO Colette Kress stated on May 22 that NVIDIA’s data center revenue for the first quarter (February to April) surged 427% year-over-year to USD 22.6 billion, primarily due to shipments of Hopper GPUs, including the H100.

On May 22, during the earnings call, Kress also mentioned that Facebook’s parent company, Meta Platforms, announced the launch of its latest large language model (LLM), “Lama 3,” which utilized 24,000 H100 GPUs. This was the highlight of Q1. She also noted that major cloud computing providers contributed approximately “mid-40%” of NVIDIA’s data center revenue.

NVIDIA CEO Jensen Huang also stated in the call, “We see increasing demand of Hopper through this quarter,” adding that he expects demand to outstrip supply for some time as NVIDIA transitions to Blackwell.

As per a report from MoneyDJ, Wall Street had previously been concerned that NVIDIA’s customers might delay purchases while waiting for the Blackwell series. Sources cited by the report predict that the Blackwell chips will be delivered in the fourth quarter of this year.

NVIDIA’s Q1 (February to April) financial result showed that revenue soared 262% year-over-year to USD 26.04 billion, with adjusted earnings per share at USD 6.12. Meanwhile, NVIDIA’s data center revenue surged 427% year-over-year to USD 22.6 billion.

During Q1, revenue from networking products (mainly Infiniband) surged more than threefold to USD 3.2 billion compared to the same period last year. Revenue from gaming-related products increased by 18% year-over-year to USD 2.65 billion. Looking ahead to this quarter (May to July), NVIDIA predicts revenue will reach USD 28 billion, plus or minus 2%.

NVIDIA’s adjusted gross margin for Q1 was 78.9%. The company predicts that this quarter’s adjusted gross margin will be 75.5%, plus or minus 50 basis points. In comparison, competitor AMD’s gross margin for the first quarter was 52%.

Read more

• [News] CoWoS Production Capacity Reportedly Falls Short of GPU Demand
• [News] NVIDIA’s H100 Chip Heat Declines, CoWoS Capacity Eases

(Photo credit: NVIDIA)

Memory giants Samsung, SK Hynix, and Micron are all actively investing in high-bandwidth memory (HBM) production. Industry sources cited in a report from Commercial Times indicate that due to capacity crowding effects, DRAM products may face shortages in the second half of the year.

According to TrendForce, the three largest DRAM suppliers are increasing wafer input for advanced processes. Following a rise in memory contract prices, companies have boosted their capital investments, with capacity expansion focusing on the second half of this year. It is expected that wafer input for 1alpha nm and above processes will account for approximately 40% of total DRAM wafer input by the end of the year.

HBM production will be prioritized due to its profitability and increasing demand. Regarding the latest developments in HBM, TrendForce indicates that HBM3e will become the market mainstream this year, with shipments concentrated in the second half of the year.

Currently, SK Hynix remains the primary supplier, along with Micron, both utilizing 1beta nm processes and already shipping to NVIDIA. Samsung, using a 1alpha nm process, is expected to complete qualification in the second quarter and begin deliveries mid-year.

The growing content per unit in PCs, servers, and smartphones is driving up the consumption of advanced process capacity each quarter. Servers, in particular, are seeing the highest capacity increase—primarily driven by AI servers with content of 1.75 TB per unit. With the mass production of new platforms like Intel’s Sapphire Rapids and AMD’s Genoa, which require DDR5 memory, DDR5 penetration is expected to exceed 50% by the end of the year.

As HBM3e shipments are expected to be concentrated in the second half of the year—coinciding with the peak season for memory demand—market demand for DDR5 and LPDDR5(X) is also expected to increase. With a higher proportion of wafer input allocated to HBM production, the output of advanced processes will be limited. Consequently, capacity allocation in the second half of the year will be crucial in determining whether supply can meet demand.

Samsung expects existing facilities to be fully utilized by the end of 2024. The new P4L plant is slated for completion in 2025, and the Line 15 facility will undergo a process transition from 1Y nm to 1beta nm and above.

The capacity of SK Hynix’s M16 plant is expected to expand next year, while the M15X plant is also planned for completion in 2025, with mass production starting at the end of next year.

Micron’s facility in Taiwan will return to full capacity next year, with future expansions focused on the US. The Boise facility is expected to be completed in 2025, with equipment installations following and mass production planned for 2026.

With the expected volume production of NVIDIA’s GB200 in 2025, featuring HBM3e with 192/384GB specifications, HBM output is anticipated to nearly double. Each major manufacturer will invest in HBM4 development, prioritizing HBM in their capacity planning. Consequently, due to capacity crowding effects, there may be shortages in DRAM supply.

Read more

• HBM3e Production Surge Expected to Make Up 35% of Advanced Process Wafer Input by End of 2024, Says TrendForce
• [News] DRAM Price Surge Halts, Could HBM Demand Drive Price Increases?

(Photo credit: Samsung)

According to reports from Korean news outlet FN News and Wccftech, aiming to win back NVDIA as a major customer, Samsung has made it a priority to secure chip order from the GPU heavyweight this year. To achieve this, Samsung is reportedly doing everything possible to ensure the company’s 3nm process node, which uses GAA (Gate-All-Around) architecture, meets NVIDIA’s requirements.

Sources quoted by the reports indicated that Samsung has implemented an internal strategy called “Nemo,” specifically targeting NVIDIA. Its foundry now plans to commence mass production of the 3nm GAA process in the first half of 2024. The GAA technology is expected to overcome significant bottlenecks associated with the previous FinFET processes, but it is still uncertain if this will be sufficient to persuade NVIDIA.

NVIDIA has been cooperating with TSMC in advanced process nodes for developing its GPUs for quite a while, both in consumer and data center markets. The tech giant’s latest GPU families, including Ada Lovelace, Hopper, and Blackwell, are all manufactured using TSMC’s 5nm (4N) processes, according to the aforementioned reports.

It’s important to note that NVIDIA last used Samsung’s 8nm process for its GeForce RTX 30 “Ampere” GPUs, designed for the gaming segment. However, the successor to Ampere, the Ada Lovelace “GeForce RTX 40,” switched to TSMC’s 5nm process.

Considering the high demand for NVIDIA’s GPUs, the chipmaker is expected to procure chips from multiple semiconductor fabs, which is simliar to its previous strategy of dual-sourcing HBM and packaging materials, according to Wccftech.

(Photo credit: Samsung)

The world’s four major CSPs (Cloud Service Providers) – Microsoft, Google, Amazon, and META – are continuously expanding their AI infrastructure, with their combined capital expenditures projected to reach USD 170 billion this year. According to the industry sources cited in a report from Commercial Times, it’s pointed out that due to the surge in demand for AI chips and the increased area of silicon interposers, the number of chips that can be produced from a single 12-inch wafer is decreasing. This situation is expected to cause the CoWoS (Chip on Wafer on Substrate) production capacity under TSMC to remain in short supply.

Regarding CoWoS, according to TrendForce, the introduction of NVIDIA’s B series, including GB200, B100, B200, is expected to consume more CoWoS production capacity. TSMC has also increased its demand for CoWoS production capacity for the entire year of 2024, with estimated monthly capacity approaching 40,000 by the year-end, compared to an increase of over 150% from the total capacity in 2023. A possibility exists for the total production capacity to nearly double in 2025.

However, with NVIDIA releasing the B100 and B200, the interposer area used by a single chip will be larger than before, meaning the number of interposers obtained from a 12-inch wafer will further decrease, resulting in CoWoS production capacity being unable to meet GPU demand. Meanwhile, the number of HBM units installed is also multiplying.

Moreover, in CoWoS, multiple HBMs are placed around the GPU, and HBMs are also considered one of the bottlenecks. Industry sources indicate that HBM is a significant challenge, with the number of EUV (Extreme Ultraviolet Lithography) layers gradually increasing. For example, SK Hynix, which holds the leading market share in HBM, applied a single EUV layer during its 1α production phase. Starting this year, the company is transitioning to 1β, potentially increasing the application of EUV by three to four times.

In addition to the increased technical difficulty, the number of DRAM units within HBM  has also increased with each iteration. The number of DRAMs stacked in HBM2 ranges from 4 to 8, while HBM3/3e increases this to 8 to 12, and HBM4 will further raise the number of stacked DRAMs to 16.

Given these dual bottlenecks, overcoming these challenges in the short term remains difficult. Competitors are also proposing solutions; for instance, Intel is using rectangular glass substrates to replace 12-inch wafer interposers. However, this approach requires significant preparation, time, and research and development investment, and breakthroughs from industry players are still awaited.

Read more

• [News] TSMC’s Advanced Packaging Capacity Fully Booked by NVIDIA and AMD Through Next Year
• [Insights] Big Four CSPs Continue to Shine in Q1 2024 Financial Reports, AI Returns Garnering Attention

(Photo credit: NVIDIA)

“The Dawn of Generative AI Has Come!” This new chapter in the course of human technological evolution was first introduced by NVIDIA’s founder, Jensen Huang. Qualcomm’s CEO, Cristiano Amon, also shares this optimism regarding generative AI. Amon believes this technology is rapidly evolving and being adopted for applications such as mobile devices. It is expected to have the potential to radically transform the landscape of the smartphone industry. Similarly, Intel has declared the arrival of the “AI PC” era, signaling a major shift in computing-related technologies and applications.

This year, AI is transitioning from cloud computing to on-premise computing. Various “AI PCs” and “AI smartphones” are being introduced to the market, offering a wide range of selections. The current year of 2024 is even being referred to as the “Year of AI PC,” with brands such as Asus, Acer, Dell, Lenovo, and LG actively releasing new products to capture market share. With the rapid rise of AI PCs and AI smartphones, revolutionary changes are expected to occur in workplaces and people’s daily lives. Furthermore, the PC and smartphone industries are also expected to be reinvigorated with new sources of demand.

An AI PC refers to a laptop (notebook) computer capable of performing on-device AI computations. Its main difference from regular office or business laptops lies in its CPU, which includes an additional neural processing unit (NPU). Examples of AI CPUs include Intel’s Core Ultra series and AMD’s Ryzen 8040 series. Additionally, AI PCs come with more DRAM to meet the demands of AI computations, thereby supporting related applications like those involving machine learning.

Microsoft’s role is crucial in this context, as the company has introduced a conversational AI assistant called “Copilot” that aims to seamlessly integrate itself into various tasks, such as working on Microsoft Office documents, video calls, web browsing, and other forms of collaborative activities. With Copilot, it is now possible to add a direct shortcut button for AI on the keyboard, allowing PC users to experience a holistic collaborative relationship with AI.

In the future, various computer functions will continue to be optimized with AI. Moreover, barriers that existed for services such as ChatGPT, which still require an internet connection, are expected to disappear. Hence, AI-based apps on PCs could one day be run offline. Such a capability is also one of the most eagerly awaited features among PC users this year.

Surging Development of LLMs Worldwide Has Led to a Massive Increase in AI Server Shipments

AI-enabled applications are not limited to PCs and smartphones. For example, an increasing number of cloud companies have started providing services that leverage AI in various domains, including passenger cars, household appliances, home security devices, wearable devices, headphones, cameras, speakers, TVs, etc. These services often involve processing voice commands and answering questions using technologies like ChatGPT. Going forward, AI-enabled applications will become ubiquitous in people’s daily lives.

Not to be overlooked is the fact that, as countries and multinational enterprises continue to develop their large language models (LLMs), the demand for AI servers will increase and thus promote overall market growth. Furthermore, edge AI servers are expected to become a major growth contributor in the future as well. Small-sized businesses are more likely to use LLMs that are more modest in scale for various applications. Therefore, they are more likely to consider adopting lower-priced AI chips that also offer excellent cost-to-performance ratios.

TrendForce projects that shipments of AI servers, including models equipped with GPUs, FPGAs, and ASICs, will reach 1.655 million units in 2024, marking a growth of 40.2% compared with the 2023 figure. Furthermore, the share of AI servers in the overall server shipments for 2024 is projected to surpass 12%.

Regarding the development of AI chips in the current year of 2024, the focus is on the competition among the B100, MI300, and Gaudi series respectively released by NVIDIA, AMD, and Intel. Apart from these chips, another significant highlight of this year is the emergence of in-house designed chips or ASICs from cloud service providers.

In addition to AI chips, the development of AI on PCs and smartphones is certainly another major driving force behind the technology sector in 2024. In the market for CPUs used in AI PCs, Intel’s Core Ultra series and AMD’s Ryzen 8000G series are expected to make a notable impact. The Snapdragon X Elite from Qualcomm has also garnered significant attention as it could potentially alter the competitive landscape in the near future.

Turning to the market for SoCs used in AI smartphones, the fierce competition between Qualcomm’s Snapdragon 8 Gen 3 and MediaTek’s Dimensity 9300 series is a key indicator. Another development that warrants attention is the adoption of AI chips in automotive hardware, such as infotainment systems and advanced driver assistance systems. The automotive market is undoubtedly one of the main battlegrounds among chip suppliers this year.

The supply chain in Taiwan has played a crucial role in providing the hardware that supports the advancement of AI-related technologies. When looking at various sections of the AI ecosystem, including chip manufacturing as well as the supply chains for AI servers and AI PCs, Taiwan-based companies have been important contributors.

Taiwan-based Companies in the Supply Chain Stand Ready for the Coming Wave of AI-related Demand

In the upstream of the supply chain, semiconductor foundries and OSAT providers such as TSMC, UMC, and ASE have always been key suppliers. As for ODMs or OEMs, companies including Wistron, Wiwynn, Inventec, Quanta, Gigabyte, Supermicro, and Foxconn Industrial Internet have become major participants in the supply chains for AI servers and AI PCs.

In terms of components, AI servers are notable for having a power supply requirement that is 2-3 times greater than that of general-purpose servers. The power supply units used in AI servers are also required to offer specification and performance upgrades. Turning to AI PCs, they also have higher demands for both computing power and energy consumption. Therefore, advances in the technologies related to power supply units represent a significant indicator this year with respect to the overall development of AI servers and AI PCs. Companies including Delta Electronics, LITE-ON, AcBel Polytech, CWT, and Chicony are expected to make important contributions to the upgrading and provisioning of power supply units.

Also, as computing power increases, heat dissipation has become a pressing concern for hardware manufacturers looking to further enhance their products. The advancements in heat dissipation made by solution providers such as Sunon, Auras, AVC, and FCN during this year will be particularly noteworthy.

Besides the aforementioned companies, Taiwan is also home to numerous suppliers for other key components related to AI PCs. The table below lists notable component providers operating on the island.

With the advent of generative AI, the technology sector is poised for a boom across its various domains. From AI PCs to AI smartphones and a wide range of smart devices, this year’s market for electronics-related technologies is characterized by diversity and innovation. Taiwan’s supply chain plays a vital role in the development of AI PCs and AI servers, including chips, components, and entire computing systems. As competition intensifies in the realm of LLMs and AI chips, this entire market is expected to encounter more challenges and opportunities.

Join the AI grand event at Computex 2024, alongside CEOs from AMD, Intel, Qualcomm, and ARM. Discover more about this expo! https://bit.ly/44Gm0pK

(Photo credit: Qualcomm)