According to a report from the South Korean newspaper “Korea Joongang Daily,” following Micron’s initiation of mass production of the latest high-bandwidth memory HBM3e in February 2024, it has recently secured an order from NVIDIA for the H200 AI GPU. It is understood that NVIDIA’s upcoming H200 processor will utilize the latest HBM3e, which are more powerful than the HBM3 used in the H100 processor.

The same report further indicates that Micron secured the H200 order due to its adoption of 1b nanometer technology in its HBM3e, which is equivalent to the 12-nanometer technology used by SK Hynix in producing HBM. In contrast, Samsung Electronics currently employs 1a nanometer technology, which is equivalent to 14-nanometer technology, reportedly lagging behind Micron and SK Hynix.

The report from Commercial Times indicates that Micron’s ability to secure the NVIDIA order for H200 is attributed to the chip’s outstanding performance, energy efficiency, and seamless scalability.

As per a previous report from TrendForce, starting in 2024, the market’s attention will shift from HBM3 to HBM3e, with expectations for a gradual ramp-up in production through the second half of the year, positioning HBM3e as the new mainstream in the HBM market.

TrendForce reports that SK Hynix led the way with its HBM3e validation in the first quarter, closely followed by Micron, which plans to start distributing its HBM3e products toward the end of the first quarter, in alignment with NVIDIA’s planned H200 deployment by the end of the second quarter.

Samsung, slightly behind in sample submissions, is expected to complete its HBM3e validation by the end of the first quarter, with shipments rolling out in the second quarter. With Samsung having already made significant strides in HBM3 and its HBM3e validation expected to be completed soon, the company is poised to significantly narrow the market share gap with SK Hynix by the end of the year, reshaping the competitive dynamics in the HBM market.

Read more

• [News] Micron Begins Mass Production of HBM3e for NVIDIA’s H200
• [News] H200 Unveiled: NVIDIA Integrates HBM3e for Enhanced AI Performance

(Photo credit: Micron)

As per Chinese media mydrivers’ report, it has indicated that NVIDIA updated the EULA terms in the CUDA 11.6 installer, explicitly prohibiting third-party GPU companies from seamlessly integrating CUDA software. This move has reportedly stirred up discussions among the market.

NVIDIA updated the EULA agreement for CUDA 11.6, explicitly stating, “You may not reverse engineer, decompile or disassemble any portion of the output generated using SDK elements for the purpose of translating such output artifacts to target a non-NVIDIA platform.”

This move is reportedly speculated to target third-party projects like ZLUDA, involving Intel and AMD, as well as compatibility solutions from Chinese firms like Denglin Technology and MetaX Technology.

In response, MooreThreads issued an official statement, confirming that its MUSA and MUSIFY technologies remain unaffected. MUSA and MUSIFY are not subject to NVIDIA EULA terms, ensuring developers can use them with confidence.

In fact, since 2021, NVIDIA has prohibited other hardware platforms from running CUDA software analog layers but only warned about it in the online EULA user agreement. Currently, NVIDIA has not explicitly named any parties, issuing warnings in the agreement without taking further action, although further measures cannot be ruled out in the future.

In the past, CUDA development and its ecosystem were widely regarded as NVIDIA’s moat. However, processor architect Jim Keller previously criticized CUDA on the X platform, stating, “CUDA is a swamp, not a moat,” adding, “CUDA is not beautiful. It was built by piling on one thing at a time.”

Meanwhile, according to the account rgznai100 posting on Chinese blog CSDN, NVIDIA’s actions will have a significant impact on AI chip/GPGPU companies that previously adopted CUDA-compatible solutions. NVIDIA may initially resort to legal measures, such as lawsuits, against GPGPU companies following similar paths.

Therefore, Chinese enterprises should endeavor to enhance collaboration with the open-source community to establish an open AI compilation ecosystem, reducing the risks posed by NVIDIA’s market dominance.

Read more

• [News] HBM Manufacturers Encounter Challenges in NVIDIA Quality Tests, Raising Concerns over Yield and Production
• [News] NVIDIA Reportedly Overwhelms TSMC with 3 and 4-Nanometer Orders

(Photo credit: NVIDIA)

AMD has encountered resistance from the United States when attempting to sell a custom-designed AI chip to the Chinese market, according to Bloomberg citing sources.

The same source further revealed that AMD sought approval from the U.S. Department of Commerce to sell this AI chip to Chinese customers. The chip’s performance is lower than AMD’s products sold in other regions of China, and its design complies with U.S. export restrictions.

However, U.S. officials informed AMD that the chip’s performance is still too robust, requiring the company to obtain approval from the U.S. Department of Commerce’s Bureau of Industry and Security before sales can proceed. It remains unclear whether AMD is currently seeking a license.

The U.S. government introduced initial export control measures in 2022 and bolstered a series of measures in October 2023, encompassing additional technologies and reforming potential sales that could undermine intermediary countries/regions.

The strict limitations on the sale of NVIDIA’s chips specifically designed for China align with the initial export regulations from 2022. Subsequently, the company developed new customized but lower-capacity products for the Chinese market, aligning with the restrictions imposed by the United States in 2023.

The U.S. ban in 2022 prevented both NVIDIA and AMD from selling their most powerful artificial intelligence chips to China. When the restrictions took effect in 2022, AMD anticipated that they would not substantially affect its operations.

While AMD has not publicly discussed its development of new AI chips for China, NVIDIA, on the other hand, immediately introduced improved products with reduced performance.

NVIDIA is set to commence pre-orders for its AI chip H20 specially designed for the Chinese market by the end of March this year in response to US export bans, according to sources cited by a report from STAR Market Daily.

Still, AMD is now actively expanding into the AI chip market. In December 2023, it launched the new MI300 series, challenging NVIDIA’s chips. According to sources cited by Bloomberg’s report, this customized product for China is known as the MI309. It is still unclear which Chinese customer wishes to purchase AMD’s AI chips, which could affect the company’s authorization.

Read more

• [News] NVIDIA and AMD Clash in AI Chip Market, as TSMC Dominates Orders with Strong Momentum in Advanced Processes
• [News] NVIDIA’s H100 AI Chip No Longer Out of Reach, Inventory Pressure Reportedly Forces Customers to Resell

(Photo credit: AMD)

The surge in demand for NVIDIA’s AI processors has made High Bandwidth Memory (HBM) a key product that memory giants are eager to develop. However, according to South Korean media DealSite cited by Wccftech on March 4th, the complex architecture of HBM has resulted in low yields, making it difficult to meet NVIDIA’s testing standards and raising concerns about limited production capacity.

The report has further pointed out that HBM manufacturers like Micron and SK Hynix are grappling with low yields. They are engaged in fierce competition to pass NVIDIA’s quality tests for the next-generation AI GPU.

The yield of HBM is closely tied to the complexity of its stacking architecture, which involves multiple memory layers and Through-Silicon Via (TSV) technology for inter-layer connections. These intricate techniques increase the probability of process defects, potentially leading to lower yields compared to simpler memory designs.

Furthermore, if any of the HBM chips are defective, the entire stack is discarded, resulting in inherently low production yields. As per the source cited by Wccftech, it has indicated that the overall yield of HBM currently stands at around 65%, and attempts to improve yield may result in decreased production volume.

Micron announced on February 26th the commencement of mass production of High Bandwidth Memory “HBM3e,” to be used in NVIDIA’s latest AI chip “H200” Tensor Core GPU. The H200 is scheduled for shipment in the second quarter of 2024, replacing the current most powerful H100.

On the other hand, Kim Ki-tae, Vice President of SK Hynix, stated on February 21st in an official blog post that while external uncertainties persist, the memory market is expected to gradually heat up this year. Reasons include the recovery in product demand from global tech giants. Additionally, the application of AI in devices such as PCs or smartphones is expected to increase demand not only for HBM3e but also for products like DDR5 and LPDDR5T.

Kim Ki-tae pointed out that all of their HBM inventory has been sold out this year. Although it’s just the beginning of 2024, the company has already begun preparations for 2025 to maintain its market-leading position.

Per a previous TrendForce press release, the three major original HBM manufacturers held market shares as follows in 2023: SK Hynix and Samsung were both around 46-49%, while Micron stood at roughly 4-6%.

Read more

• [News] Micron Begins Mass Production of HBM3e for NVIDIA’s H200
• [News] SK Hynix Announces Complete Sales of HBM for the Year, Memory Market Poised for Recovery

(Photo credit: SK Hynix)

NVIDIA has yet to officially announce the exact release dates for its next-generation AI chip architectures, the Blackwell GPU, and the B100 chip. However, Dell’s Chief Operating Officer, Jeff Clarke, recently revealed ahead of schedule during Dell’s Q4 2024 Earnings Call that NVIDIA is set to introduce the Blackwell architecture next year, with plans to release not only the B100 chip but also another variant, the B200 chip.

Following Dell’s recent financial report, Clarke disclosed in a press release that NVIDIA is set to unveil the B200 product featuring the Blackwell architecture in 2025.

Clarke also mentioned that Dell’s flagship product, the PowerEdge XE9680 rack server, utilizes NVIDIA GPUs, making it the fastest solution in the company’s history. He expressed anticipation for NVIDIA’s release of the B100 and B200 chips. This news has sparked significant market interest, as NVIDIA has yet to publicly mention the B200 chip.

Clarke further stated that the B200 chip will showcase Dell’s engineering expertise in high-end servers, especially in liquid cooling systems. As for the progress of the B100 chip, NVIDIA has yet to disclose its specific parameters and release date.

NVIDIA’s current flagship H200 chip in the high-performance computing market adopts the Hopper GPU architecture paired with HBM3e memory chips, considered the most capable chip for AI computing in the industry.

However, NVIDIA continues to accelerate the development of its next-generation AI chip architectures. According to NVIDIA’s previously disclosed development roadmap, the next-generation product after the H200 chip is the B100 chip. Therefore, the expectation was that the B100 chip would be the highest-specification chip based on the Blackwell GPU architecture. Nevertheless, with the emergence of the B200 chip, it has sparked further speculation.

Previously, media speculation cited by the report from Commercial Times stated based on the scale of the H200 chip that the computational power of the B100 chip would be at least twice that of the H200 and four times that of the H100.

Read more

• [News] NVIDIA Expands Advanced Packaging Supply Chain with Intel’s Inclusion, Diverting Orders from TSMC
• [News] NVIDIA Reportedly Overwhelms TSMC with 3 and 4-Nanometer Orders

(Photo credit: NVIDIA)