On November 13, NVIDIA unveiled the AI computing platform HGX H200, featuring the Hopper architecture, equipped with H200 Tensor Core GPU and high-end memory to handle the vast amounts of data generated by AI and high-performance computing.

This marks an upgrade from the previous generation H100, with a 1.4x increase in memory bandwidth and a 1.8x increase in capacity, enhancing its capabilities for processing intensive generative AI tasks.

The internal memory changes in H200 represent a significant upgrade, as it adopts the HBM3e for the first time. This results in a notable increase in GPU memory bandwidth, soaring from 3.35TB per second in H100 to 4.8TB per second.

The total memory capacity also sees a substantial boost, rising from 80GB in H100 to 141GB. When compared to H100, these enhancements nearly double the inference speed for the Llama 2 model.

H200 is designed to be compatible with systems that already support H100, according to NVIDIA. The company states that cloud service providers can seamlessly integrate H200 into their product portfolios without the need for any modifications.

This implies that NVIDIA’s server manufacturing partners, including ASRock, ASUS, Dell, Eviden, GIGABYTE, HPE, Ingrasys, Lenovo, Quanta Cloud, Supermicro, Wistron, and Wiwynn, have the flexibility to replace existing processors with H200.

The initial shipments of H200 are expected in the second quarter of 2024, with cloud service giants such as Amazon, Google, Microsoft, and Oracle anticipated to be among the first to adopt H200.

What is HBM?

“The integration of faster and more extensive HBM memory serves to accelerate performance across computationally demanding tasks including generative AI models and [high-performance computing] applications while optimizing GPU utilization and efficiency,” said Ian Buck, the Vice President of High-Performance Computing Products at NVIDIA.

What is HBM? HBM refers to stacking DRAM layers like building blocks and encapsulating them through advanced packaging. This approach increases density while maintaining or even reducing the overall volume, leading to improved storage efficiency.

TrendForce reported that the HBM market’s dominant product for 2023 is HBM2e, employed by the NVIDIA A100/A800, AMD MI200, and most CSPs’ (Cloud Service Providers) self-developed accelerator chips.

As the demand for AI accelerator chips evolves, in 2023, the mainstream demand is projected to shift from HBM2e to HBM3, with estimated proportions of approximately 50% and 39%, respectively.

As the production of acceleration chips utilizing HBM3 increases gradually, the market demand in 2024 is expected to significantly transition to HBM3, surpassing HBM2e directly. The estimated proportion for 2024 is around 60%.

Since Manufacturers plan to introduce new HBM3e products in 2024, HBM3 and HBM3e are expected to become mainstream in the market next year.

TrendForce clarifies that the so-called HBM3 in the current market should be subdivided into two categories based on speed. One category includes HBM3 running at speeds between 5.6 to 6.4 Gbps, while the other features the 8 Gbps HBM3e, which also goes by several names including HBM3P, HBM3A, HBM3+, and HBM3 Gen2.

HBM3e will be stacked with 24Gb mono dies, and under the 8-layer (8Hi) foundation, the capacity of a single HBM3e will jump to 24GB.

According to the TrendForce’s previous news release, the three major manufacturers currently leading the HBM competition – SK hynix, Samsung, and Micron – have the following progress updates.

SK hynix and Samsung began their efforts with HBM3, which is used in NVIDIA’s H100/H800 and AMD’s MI300 series products. These two manufacturers are expected to sample HBM3e in Q1 2024 previously. Meanwhile, Micron chose to skip HBM3 and directly develop HBM3e.

However, according to the latest TrendForce survey, as of the end of July this year, Micron has already provided NVIDIA with HBM3e verification, while SK hynix did so in mid-August, and Samsung in early October.

(Image: Nvidia)

Rumors swirl around AMD’s upcoming chip architecture, codenamed “Prometheus,” featuring the Zen 5C core. As reported by TechNews, the chip is poised to leverage both TSMC’s 3nm and Samsung’s 4nm processes simultaneously, marking a shift in the competitive landscape from process nodes, yield, and cost to factors like capacity, ecosystem, and geopolitics, are all depends on customer considerations.

Examining yields, TSMC claims an estimated 80% yield for its 4nm process, while Samsung has surged from 50% to an impressive 75%, aligning with TSMC’s standards and raising the likelihood of chip customers returning. Speculation abounds that major players such as Qualcomm and Nvidia may reconsider their suppliers, with industry sources suggesting Samsung’s 4nm capacity is roughly half of TSMC’s.

Revegnus, a reputable X(formerly Twitter) source, unveiled information from high-level Apple meetings, indicating a 63% yield for TSMC’s 3nm process but at double the price of the 4nm process. In the 4nm realm, Samsung’s yield mirrors TSMC’s, with Samsung showing a faster-than-expected yield recovery.

Consequently, with Samsung’s significant improvements in yield and capacity, coupled with TSMC’s decision to raise prices, major clients may explore secondary suppliers to diversify outsourcing orders, factoring in considerations such as cost and geopolitics. Recent reports suggest Samsung is in final negotiations for a 4nm collaboration with AMD, planning to shift some 4nm processor orders from TSMC to Samsung.

Beyond AMD, the Tensor G3 processor in Google’s Pixel 8 series this year adopts Samsung’s 4nm process. Samsung’s new fabs in Taylor, Texas, sees its inaugural customer in its Galaxy smartphones, producing Exynos processors.

Furthermore, Samsung announced that U.S.-based AI solution provider Groq will entrust the company to manufacture next-generation AI chips using the 4nm process, slated to commence production in 2025, marking the first order for the new Texas plant.

Regarding TSMC’s 4nm clients, alongside longstanding partners like Apple, Nvidia, Qualcomm, MediaTek, AMD, and Intel, indications propose a potential transition to TSMC’s 4nm process for Tensor G4, while Tensor G5 will be produced using TSMC’s 3nm process. Ending the current collaboration with Samsung, TSMC’s chip manufacturing debut is anticipated to be delayed until 2025.

Last year, rumors circulated about Tesla, the electric vehicle giant, shifting orders for the 5th generation self-driving chip, Hardware 5 (HW 5.0), to TSMC. This decision was prompted by Samsung’s lagging 4nm process yield at that time. However, with Samsung’s improved yield, industry inclination leans towards splitting orders between the two companies.

According to MoneyDJ’s report, Samsung Electronics, the South Korean smartphone giant, unveiled its latest foldable phones, the Galaxy Z Fold5 and Galaxy Z Flip5, in August. With a year until the next generation hits the market, speculation is arising that Samsung plans to incorporate foldable features into mid-range models. This move aims to lower the entry barrier, attract a broader customer base, and strengthen Samsung’s leading position in the foldable phone market.

TrendForce recently reported that Android smartphone brands are actively entering the foldable phone market, aiming to break through the plateau in smartphone market growth with the unique design of foldable phones. However, the widespread adoption of foldable phones faces a significant obstacle in their high pricing.

According to supply chain sources, Samsung is set to launch a mid-range foldable phone in 2024, targeting a relatively budget-friendly price range of $400 to $500 USD.

In August, Samsung launched its latest generation of foldable phones, the Galaxy Z Fold5 and Galaxy Z Flip5, maintaining a premium pricing strategy. The suggested retail prices are $1,799 USD for the Galaxy Z Fold5 and $999 USD for the Galaxy Z Flip5.

The market is eagerly anticipating Samsung’s introduction of a mid-range foldable phone. However, as of now, this remains in the speculative phase, and there’s no information available regarding its design, specifications, or other details.

Previous market rumors suggested that Samsung’s Z series of foldable phones might follow the flagship S series by introducing a “Lite Flagship” FE version. This version is expected to feature hardware downgrades to offer a more budget-friendly price, aiming to attract consumers.

According to a TrendForce’s forecast, as foldable phones gain increased acceptance in the consumer market, the global shipment volume of foldable smartphones is estimated to reach 18.3 million units in 2023. This represents a substantial 43% growth compared to 2022, although it accounts for only 1.6% of the total smartphone market sales. Looking ahead to 2024, the shipment volume is expected to grow by another 38%, reaching 25.2 million units, and the market share is projected to increase to 2.2%.

Read more

• [News] Can Foldable Phones Become Lifeline for the Declining Global Smartphone Market?
• After the iPhone 15 Series Debut, Will a Foldable Phone Be Apple’s Next Step?

(Photo credit: Samsung)

Samsung’s upcoming flagship Galaxy S24 series, reportedly set to be unveiled in mid-January 2024, will feature Qualcomm’s latest Snapdragon 8 Gen 3 processor. This new series will highlight Samsung’s first-ever generative AI smartphone, with a sales target expected to increase by over 10% compared to the previous S23 model, reaching more than 34 million units.

As reported by “The Korea Daily,” Samsung is planning to unveil the Galaxy S24 series of new smartphones in mid-January 2024, making it the first major Android flagship to be launched next year. To ensure the successful market debut of the new devices, Samsung is expected to initiate the procurement process from its supply chain in the very near future.

Industry sources have revealed that Samsung’s internal estimates project smartphone sales for next year to reach approximately 253 million units and this figure is expected to be adjusted upwards in the near future according to Economic Daily.

In comparison, Apple’s projected total iPhone sales for the next year, according to a market analyst, are expected to be around 250 million units, making the estimates of these two major smartphone giants quite close.

It is reported that Samsung’s recent flagship devices in the S series have delivered subpar performance. Both the S21 and S22 models achieved sales figures of fewer than 30 million units, while the estimated sales for the S23 series stand at around 31 million units.

Notably, Samsung is currently collaborating with major players such as Google and Microsoft, and there is a strong likelihood that they will incorporate generative AI features like Google Bard or ChatGPT into the Galaxy S24.

This move aims to make the Galaxy S24 their first generative AI smartphone, serving as a prominent selling point for the new device and enhancing the operating system, thereby narrowing the gap with Apple.

Industry experts point out that while smartphone brands tend to estimate sales figures for new products before their launch, the smartphone market has experienced weakened demand in recent years.

Many brands have adopted a conservative approach to sales estimates, even controlling production capacity and marketing budgets to reduce risks. In this context, Samsung’s decision to set clear growth targets stands as a demonstration of their high confidence in the new product.

As for the supply chain, Largan Precision is the primary supplier of main camera components for the Galaxy S24, while GIS is responsible for supplying the in-display fingerprint module, and TXC Corporation provides quartz components.

(Photo credit: Samsung)

On November 7th, Intel held its “Intel Innovation Taipei 2023 Technology Forum”, with CEO Pat Gelsinger highlighting the healthy state of PC inventory. He also expressed optimism about the injection of several more years of innovative applications and evolution in PCs through AI. 

Intel Aims to Ship over One Hundred Million AI PC within the Next Two Years

Gelsinger expressed that the PC inventory has reached a healthy level, and he is optimistic about the future growth of AI PCs, which are equipped with AI processors or possess AI computing capabilities. He anticipates that AI will be a crucial turning point for the PC industry.

Additionally, Gelsinger stated that the server industry may have seemed uneventful in recent years, but with the accelerated development of AI, it has become more exciting. AI is becoming ubiquitous, transitioning from the training phase to the deployment phase, and various platforms will revolve around AI.

Gelsinger expressed his strong confidence in Intel’s position in the AI PC market, expecting to ship over one hundred million units within two years.

Intel’s Ambitious Expansion in Semiconductor Foundry Landscape

Intel is actively promoting its IDM 2.0 strategy, with expectations from the industry that the company, beyond its brand business, has advanced packaging capabilities to support semiconductor foundry operations. In the future, Intel is poised to compete with rivals such as TSMC and Samsung.

Gelsinger noted that some have viewed Intel’s plan of achieving five technical nodes in four years as “an ambitious endeavor.” However, he emphasized that Intel remains committed to its original goal of advancing five process nodes within four years.

The company’s foundry business has received positive responses from numerous potential customers, and while it may take three to four years for significant expansion, the advanced packaging aspect may only require two to three quarters to get on track.

This transformation marks a significant shift for the company, setting new standards in the industry. Intel is making steady progress in its four-year plan to advance five nodes, and Moore’s Law will continue to extend. The construction of Intel’s new factories is also ongoing.

According to Intel’s roadmap, Intel 7 and Intel 4 are already completed, Intel 3 is set for mass production in the latter half of this year, and Intel 20A and 18A are expected to enter mass production in the first and second halves of next year, respectively.

Regarding this roadmap, according to NIKKEI Asia’s report, Gelsinger also mentioned at the forum that for the 18A process, they currently have many test wafers in production. Additionally, the development of 18A has been completed, and it is progressing rapidly towards the production phase.

Read more

• [News] Intense Competition in Advancing Processes at the 2nm by Samsung, Intel, and TSMC

(Photo credit: Intel Newsroom)