In late March, NVIDIA CEO Jensen Huang personally delivered the first high-end DGX H200 AI server to customer OpenAI. According to a report from Commercial Times, following this, the upstream chips for the H200 AI-GPU entered mass production in late Q2, with large-scale deliveries expected from Q3 onwards.

Downstream companies, including Inventec, Quanta (QCT), Hewlett Packard Enterprise (HPE), Supermicro, and Lenovo, have reportedly listed the H200 products as ready for shipment, with deliveries anticipated to begin in the second half of the year.

The same report, citing sources, indicates that current pending orders are still largely focused on the H100 in the HGX architecture, with the H200’s share remaining limited. The H200 shipments expected in Q3 will primarily be NVIDIA’s DGX H200. As for the B100, there is already some visibility, with shipments expected in the first half of next year.

Despite the CoWoS production capacity catching up and a significant easing in the supply of AI GPUs, due to the allocation system, the delivery time for the main H100 GPU shipments from various end-system partners can still reach up to 20 weeks.

However, major Taiwanese manufacturers such as Quanta, Inventec, Wistron, Gigabyte, and ASUS have seen a substantial boost in their overall server operations, driven by AI server business in the first half of the year. They are reportedly optimistic about AI server shipments in the second half of the year, expecting strong demand to continue, making the server business effectively free from any off-season throughout the year.

Among them, Wistron holds a significant advantage in the production and supply of H100 series substrates and the subsequent B100 series GPU modules and substrates. Starting from the second quarter, Wistron’s AI server-related business has shown high visibility, providing strong support for its overall operations.

On the other hand, the H20 series, an AI chip tailored for China due to U.S. chip restrictions, has also seen demand in the Chinese market. As NVIDIA recently secured a rush order for the H20 series, Taiwanese companies, including Wistron and Inventec, are expected to benefit.

Read more

• [News] Taiwanese AI Server Supply Chain to Benefit from Apple’s Potential Collaboration with Baidu and Alibaba
• [News] Rumored Sharp Drop in H100 Server Black Market Prices in China Raises Concerns Over Market Stability

(Photo credit: NVIDIA)

According to a report from Nikkei citing sources, memory giant Micron Technology is building a pilot production line for advanced high-bandwidth memory (HBM) in the United States and is considering producing HBM in Malaysia for the first time to capture more demand from the AI boom.

Reported on June 19, Micron is said to be expanding its HBM-related R&D facilities at its headquarters in Boise, Idaho, which include production and verification lines. Additionally, Micron is considering establishing HBM production capacity in Malaysia, where it already operates chip testing and assembly plants.

Nikkei’s report further noted that Micron’s largest HBM production facility is located in Taichung, Taiwan, where expansion efforts are also underway. Micron is said to have set a goal to triple its HBM market share to 24-26% by the end of 2025, which would bring it close to its traditional DRAM market share of approximately 23-25%.

Earlier this month, a report from a Japanese media outlet The Daily Industrial News also indicated that Micron planned to build a new DRAM plant in Hiroshima, with construction scheduled to begin in early 2026 and aiming for completion of plant buildings and first tool-in by the end of 2027.

Per industry sources cited by TechNews, Micron is expected to invest between JPY 600 to 800 billion in the new facility, located adjacent to the existing Fab15 facility. Initially, the new plant will focus on DRAM production, excluding backend packaging and testing, with a capacity emphasis on HBM products.

Micron, along with SK Hynix, has reportedly received certification from NVIDIA to produce HBM3e for the AI chip “H200.” Samsung Electronics has not yet received approval from NVIDIA; its less advanced HBM3 and HBM2e are currently primarily supplied to AMD, Google, and Amazon.

Read more

• [News] HBM Supply Shortage Prompts Micron’s Expansion, Expected Schedule in Japan and Taiwan Revealed
• [News] Micron Slightly Raises Capital Expenditure for 2024, HBM Expected to Further Drive Revenue Growth in 2025

(Photo credit: Micron)

AMD Chairman and CEO Lisa Su unveiled the company’s latest AI chip, MI325X, at the opening of Computex Taipei on June 3. She emphasized that the MI325X boasts 30% faster computing speed compared to NVIDIA’s H200. According to a report from CNA, Su also announced that AMD plans to release a new generation of AI chips each respective year, hinting at a strong competitive stance against NVIDIA.

Lisa Su announced that the MI300 series is AMD’s fastest progressing product. The tech giant’s next-generation AI chip, MI325X, features HBM3e and is built on the CDNA 3 architecture.

According to Su, AMD MI325X outperforms NVIDIA’s H200 in performance and bandwidth, more than twice than that of NVIDIA’s H200. On the other hand, MI325X delivers 30% faster computing speed compared to the H200.

Furthermore, Su also announced that AMD will release MI350 in 2025, which will be manufactured with 3nm process, while MI400 is expected to follow, launched in 2026.

On June 3, Lisa Su stated that AMD will continue its collaboration with TSMC, advancing process technology to the 3nm and even 2nm nodes. Yet, Su did not directly address the previous market rumors suggesting that AMD might switch to Samsung’s 3nm technology.

Previously, as per a report on May 29th from The Korea Economic Daily, it has speculated that AMD is likely to become a customer of Samsung Electronics’ 3nm GAA process. Reportedly, during AMD CEO Lisa Su’s appearance at the 2024 ITF World, which was hosted by the Belgian microelectronics research center imec, Su revealed that AMD plans to use the 3nm GAA process for mass-producing next-generation chips.

Per the same report, Lisa Su stated that 3nm GAA transistors can enhance efficiency and performance, with improvements in packaging and interconnect technology. This will make AMD products more cost-effective and power-efficient. The report further addressed that, as Samsung is currently the only chip manufacturer with commercialized 3nm GAA process technology, Su’s comments were interpreted as indicating that AMD will officially collaborate with Samsung for 3nm production.

Read more

• [News] South Korean Media Suggests AMD Could Become Samsung’s 3nm GAA Process Customer
• [News] Samsung Reportedly Signs USD 3 Billion HBM3e Deal with AMD

(Photo credit: AMD)

Samsung’s HBM, according to a report from TechNews, has yet to pass certification by GPU giant NVIDIA, causing it to fall behind its competitor SK Hynix. As a result, the head of Samsung’s semiconductor division was replaced. Although Samsung denies any issues with their HBM and emphasizes close collaboration with partners, TechNews, citing market sources, indicates that Samsung has indeed suffered a setback.

Samsung invested early in HBM development and collaborated with NVIDIA on HBM and HBM2, but sales were modest. Eventually, the HBM team, according to TechNews’ report, moved to SK Hynix to develop HBM products. Unexpectedly, the surge in generative AI led to a sharp increase in HBM demand, and SK Hynix, benefitting from the trend, seized the opportunity with the help of the team.

Yet, in response to the rumors about changes in the HBM team, SK Hynix has denied the claims that SK Hynix developed HBM with the help of the Samsung team and also denied the information that Samsung’s HBM team transferred to SK Hynix. SK Hynix further emphasized the fact that SK Hynix’s HBM was developed solely by its own engineers.

Samsung’s misfortune is evident; despite years of effort, they faced setbacks just as the market took off. Samsung must now find alternative ways to catch up. The market still needs Samsung, as noted by Wallace C. Kou, President of memory IC design giant Silicon Motion.

Kou reportedly stated that Samsung remains the largest memory producer, and as NVIDIA faces a supply shortage for AI chips, the GPU giant is keen to cooperate with more suppliers. Therefore, it’s only a matter of time before Samsung supplies HBM to NVIDIA.

Furthermore, Samsung also indicated in a recent statement, addressing that they are conducting HBM tests with multiple partners to ensure quality and reliability.

In the statement, Samsung indicates that it is in the process of optimizing their products through close collaboration with its customers, with testing proceeding smoothly and as planned. As HBM is a customized memory product, it requires optimization processes in line with customers’ needs.

Samsung also states that it is currently partnering closely with various companies to continuously test technology and performance, and to thoroughly verify the quality and performance of its HBM.

On the other hand, NVIDIA has various GPUs adopting HBM3e, including H200, B200, B100, and GB200. Although all of them require HBM3e stacking, their power consumption and heat dissipation requirements differ. Samsung’s HBM3e may be more suitable for H200, B200, and AMD Instinct MI350X.

Read more

• [News] Reasons for Samsung’s HBM Chips Failing Nvidia Tests Revealed, Reportedly Due to Heat and Power Consumption Issues
• [News] HBM Boom May Lead to DRAM Shortages in the Second Half of the Year

(Photo credit: SK Hynix)

The recent rapid downturn in the black market prices of AI servers equipped with NVIDIA’s highest-tier AI chip, the H100, in China, has attracted attention, as per a report from Economic Daily News. This fluctuation, triggered by US sanctions, has reportedly prompted concerns about its impact on overall supply and demand dynamics, and whether it will further squeeze normal market mechanisms.

Industry sources cited by the same report have revealed that the prices of AI servers equipped with the H100 chip have recently plummeted on the Chinese black market. This is primarily due to the imminent launch of NVIDIA’s next-generation high-end AI chip, the H200. With the transition between old and new products, scalpers who previously hoarded H100 chips to drive up prices are now offloading their large inventories.

As per a report from Reuters, despite the US expanding its ban on AI technology-related exports to China last year, some dealers are still taking risks. There is still trading of H100 chips in the Huaqiangbei electronics market in northern Shenzhen, but it has all gone underground. The chips are said to be mainly imported into China through purchasing agents or shell companies set up overseas, making them accessible to Chinese universities, research institutions, and even companies through special dealer channels.

Due to the US ban, both the H100 chip and AI servers equipped with it can only be traded on the black market, not openly. Scalpers have significantly inflated prices, with servers featuring the H100 chip reaching over CNY 3 million (over USD 420,000) in China, compared to the official price of USD 280,000 to USD 300,000, resulting in profits of over 10% for some middlemen after deducting logistics and tariffs.

With the H200 set to launch in the second quarter, the H100 will become the “previous generation” product. Consequently, middlemen who had hoarded H100 chips are eager to sell their inventory, leading to a rapid correction in prices.

Recently, servers with the H100 chip on the China black market have dropped to around CNY 2.7 to 2.8 million, with spot prices in Hong Kong falling to around CNY 2.6 million, representing a decline of over 10%.

According to a previous report from Reuters, in response to Chinese universities and research institutions reportedly acquired high-end AI chips from NVIDIA through distributors, a NVIDIA spokesperson stated that the report does not imply that NVIDIA or any of its partners violated export control regulations, and the proportion of these products in global sales is negligible. Nvidia complies with U.S. regulatory standards.

Read more

• [News] NVIDIA’s H100 Chip Heat Declines, CoWoS Capacity Eases
• [News] NVIDIA’s H100 AI Chip No Longer Out of Reach, Inventory Pressure Reportedly Forces Customers to Resell

(Photo credit: NVIDIA)