Amid concerns on the delay of NVIDIA’s Blackwell, CEO Jensen Huang spoke at the Goldman Sachs Communacopia and Technology Conference a few hours ago, trying to ease the doubts of the market by expressing his optimism on the demand of its products and the company’s future prospects.

“We have a lot of people on our shoulders, and everybody is counting on us,” said Huang, according to a report by Yahoo Finance. He even joked that as the world relies so much on the company, the engineers may have to spend more time on work. “Less sleep is fine. Three solid hours is all we need.”

Huang also elaborated on the demand for the delivery of NVIDIA’s components, technology, infrastructure and software, stating that it is so overwhelming that people may get “emotional,” as it has a direct impact on their revenue and competitiveness.

It is worth noting that Huang also mentioned that NVIDIA heavily relies on TSMC for producing its most important chips, as in many ways, according to a report by Economic Daily News. He said TSMC’s agility and ability to respond to demand are incredible.

Huang stated that most of NVIDIA’s technology is self-developed, and if necessary, orders could be shifted to other suppliers. However, such adjustments could lead to a decline in chip quality, according to the report.

According to a previous report from Commercial Times, NVIDIA has reportedly executed changes to the Blackwell series’ GPU mask. Therefore, the process can now proceed without re-taping out, with NVIDIA’s updated version of B200 expected to be completed by late October, allowing the GB200 to enter mass production in December.

Moreover, in his latest meeting with Goldman Sachs, Huang noted that the first trillion dollars of data centers is going to get accelerated, creating a new type of software, generative AI.

Citing Huang’s remarks, the report by Yahoo Finance stated that it matters a lot because generative AI is not just a tool but a “skill,” so for the first time, the AI chip giant is developing skills that will enhance human capabilities.

According to Yahoo Finance, Huang said that NVIDIA, along with cloud service providers (CSPs), build the infrastructure in the cloud so developers can access these machines to train, fine-tune, and safeguard models.

It is worth noting that Huang tried to materialize the benefit, saying that for every dollar a CSP spends with NVIDIA, it results in USD 5 worth of rentals. He also said while training AI models is resource-intensive, it pays off in the long run.

Citing Huang, the report stated that NVIDIA’s servers may seem expensive at first glance, as it potentially costs a couple of million dollars per rack. However, they replace thousands of nodes. What is remarkable is that the cost of cables for old, general-purpose computing systems is higher than consolidating everything into a single, dense rack, Huang said.

According to Yahoo Finance, Huang also noted that the days of software engineers writing every line of code are completely behind. In his vision, every software engineer will have digital companions working alongside them 24/7.

In addition, NVIDIA, with its 32,000 employees, hopes to be supported by “100 times more digital engineers” in the near future, the report noted.

Notably, there seems to be another piece of good news for the U.S. chip giant. According to a report by Reuters, the U.S. government is said to be mulling, allowing NVIDIA to export advanced chips to Saudi Arabia, which would enable the country to train and operate the most powerful AI models.

According to the report, Saudi Arabia expects to receive shipments of NVIDIA’s most advanced chips, the H200s, which were first used in OpenAI’s GPT-4o.

Read more

• [News] NVIDIA’s Blackwell Overcomes Delays, as GB200 Reportedly Sets for December Mass Production
• [News] Foxconn Remains Unfazed Despite NVIDIA’s GB200 Delay, with Customers Aim to Secure Their Orders

(Photo credit: NVIDIA)

After its 8-Hi HBM3e entered mass production in February, Micron officially introduced the 12-Hi HBM3e memory stacks on Monday, which features a 36 GB capacity, according to a report by Tom’s Hardware. The new products are designed for cutting-edge processors used in AI and high-performance computing (HPC) workloads, including NVIDIA’s H200 and B100/B200 GPUs.

It is worth noting that the achievement has made the US memory chip giant almost on the same page with the current HBM leader, SK hynix. Citing Justin Kim, president and head of the company’s AI Infra division at SEMICON Taiwan last week, another report by Reuters notes that SK hynix is set to begin mass production of its 12-Hi HBM3e chips by the end of this month.

Samsung, on the other hand, is said to have completed NVIDIA’s quality test for the shipment of 8-Hi HBM3e memory, while the company is still working on the verification of its 12-Hi HBM3e.

Micron’s 12-Hi HBM3e memory stacks, according to Tom’s Hardware, feature a 36GB capacity, a 50% increase over the previous 8-Hi models, which had 24GB. This expanded capacity enables data centers to handle larger AI models, such as Meta AI’s Llama 2, with up to 70 billion parameters on a single processor. In addition, this capability reduces the need for frequent CPU offloading and minimizes communication delays between GPUs, resulting in faster data processing.

According to Tom’s Hardware, in terms of performance, Micron’s 12-Hi HBM3e stacks deliver over 1.2 TB/s. Despite offering 50% more memory capacity than competing products, Micron’s HBM3e consumes less power than the 8-Hi HBM3e stacks.

Regarding the future roadmap of HBM, Micron is said to be working on its next-generation memory solutions, including HBM4 and HBM4e. These upcoming memory technologies are set to further enhance performance, solidifying Micron’s position as a leader in addressing the increasing demand for advanced memory in AI processors, such as NVIDIA’s GPUs built on the Blackwell and Rubin architectures, the report states.

Read more

• [News] SK hynix to Begin 12-Stack HBM3e Mass Production, Marking Key Moment in HBM Battlefield
• [News] Samsung Likely Emerges as the Pacemaker for the AI Market if It Secures HBM3e Supply to NVIDIA

(Photo credit: Micron)

AI chip giants NVIDIA and AMD have been under heated competition for a couple of years. NVIDIA, though controls the lion’s share of the market for AI computing solutions, had been challenged by AMD while the latter launched Instinct MI300X GPU in late 2023, claiming the product to be the fastest AI chip in the world, which beats NVIDIA’s H200 GPUs.

However, months after the launch of MI300X, an analysis by Richard’s Research Blog indicates that AMD’s MI300X’s cost is significantly higher than NVIDIA’s H200’s, while H200 outperforms MI300X by over 40% regarding inference production applications, which makes NVIDIA’s high margin justifiable.

AMD’s MI300X: More Transistors, More Memory Capacity, More Advanced Packaging…with a Higher Cost

The analysis further compares the chip specifications between the two best-selling products and explores their margins. NVIDIA’s H200 is implemented using TSMC’s N4 node with 80 billion transistors. On the other hand, AMD’s MI300X is built with 153 billion transistors, featuring TSMC’s 5nm process.

Furthermore, NVIDIA’s H200 features 141GB of HBM3e, while AMD’s MI300X is equipped with 192GB of HBM3. Regarding packaging techniques, while NVIDIA is using TSMC’s CoWoS 2.5D in the H200, AMD’s MI300X has been moved to CoWoS/SoIC 3D with a total of 20 dies/stacks, which significantly increases its complexity.

According to the analysis, under the same process, the number of transistors in the logic compute die and the total die size/total cost are roughly proportional. AMD’s MI300X, equipped with nearly twice the number of transistors compared to NVIDIA’s H200, therefore, is said to cost twice as much of the latter in this respect.

With 36% more memory capacity and much higher packaging complexity, AMD’s MI300X is said to suffer a significantly higher manufacturing cost than NVIDIA’s H200. It is also worth noting that as NVIDIA is currently the dominant HBM user in the market, the company must enjoy the advantage of lower procurement costs, the analysis suggests.

This is the price AMD has to pay for the high specifications of the MI300X, the analysis observes.

NVIDIA’s 80% Margin: High at First Glance, but Actually Justifiable

On the other hand, citing the results of MLPerf tests, the analysis notes that in practical deployment for inference production applications, the H200 outperforms the MI300X by over 40%. This means that if AMD wants to maintain a similar cost/performance ratio (which CSP customers will demand), the MI300X price must be about 30% lower than the H200. The scenario does not take other factors into consideration, including NVIDIA’s familiarity with secondary vendors, the Compute Unified Device Architecture (CUDA), as well as related software.

Therefore, the analysis further suggests that NVIDIA’s 80% gross margin, though might seem to be high at first glance, actually allows room for its competitors to survive. If NVIDIA were to price its products below a 70% margin, its rivals might struggle with negative operating profits.

In addition to achieving better product performance at a lower cost through superior hardware and software technology, NVIDIA excels at non-technical economic factors, including R&D and the scaling of expensive photomasks, which impact operational expenditures (OPEX) and cost distribution as well, while its long-term commitments to its clients, confidence, and time-to-market also play a role, the analysis notes.

Regarding the key takeaways from their latest earnings reports, NVIDIA claims the demand for Hopper remains strong, while Blackwell chips will potentially generate billions of dollars in revenue in the fourth quarter. AMD’s Instinct MI300 series, on the other hand, has emerged as a primary growth driver, as it is expected to generate more than USD 4.5 billion in sales this year.

Read more

• [News] NVIDIA’s Q3 Forecast Fails to Impress, while It Claims Blackwell to Bring Several Billion Dollars Revenue in Q4
• [News] Foxconn Confirms Shipment of NVIDIA’s GB200 to Start in Q4 as Planned

(Photo credit: NVIDIA)

On August 27, AI server giant Supermicro was accused of accounting violations, inadequate disclosure of related party transactions, and evading sanctions by selling products to Russia by short-seller Hindenburg Research.

In addition, Supermicro announced on August 28 that it would delay the release of its annual report, potentially facing order withdrawals. Industry sources also believe this news presents a chance for Supermicro’s competitor Dell to gain market share.

Besides Dell, a report from Commercial Times also points out that Hewlett Packard Enterprise (HPE) could benefit from the shift in orders, potentially boosting shipments for its Taiwanese supply chain partners such as Wistron, Inventec, Quanta, and Foxconn.

The report from Commercial Times also cite sources, suggesting that this shift could provide Gigabyte, which is actively promoting its liquid-cooled products for NVIDIA’s H200 series, with  opportunities in the second half of the year.

Wistron, as a key supplier of motherboard and GPU accelerator cards for NVIDIA’s Hopper and Blackwell GPU, is not only a major supplier for Supermicro’s server motherboards but also for Dell. Its clients include HPE and Lenovo as well, which makes the company one of the primary beneficiaries.

Similarly, Inventec, one of the server motherboard suppliers, is also expected to benefit if the shift in orders boosts Dell, HPE, and Lenovo.

Moreover, one of Supermicro’s largest clients, CoreWeave, is transitioning to become a cloud computing service provider specializing in GPU-accelerated computing.

This shift has increased demand for GPU-accelerated computing and liquid cooling solutions. Reportedly, it’s believed that Gigabyte, which holds orders from CoreWeave, could be one of the biggest beneficiaries of the upcoming order shift.

Read more

• [News] Supermicro Faces Order Withdrawal Crisis, Potentially Benefiting Dell
• [News] Quick Couplings’ Buying Surge Reportedly Leads to AI Server Component Shortage Brought up by Supermicro

(Photo credit: Supermicro)

Super Micro Computer, Inc. (Supermicro), a provider of servers and storage solutions, released its financial results for the fourth quarter of the fiscal year 2024 (ending June 30, 2024) on August 6. The revenue increased by 143.6% year-over-year (37.9% quarter-over-quarter) to USD 5.31 billion. The Non-GAAP diluted earnings per share (EPS) rose by 78.1% year-over-year (decreased by 6% quarter-over-quarter) to $6.25.

Supermicro forecasts that for the first quarter of the fiscal year 2025 (ending September 30, 2024), revenue will be between USD 6 billion and 7 billion (midpoint of USD 6.5 billion), and the Non-GAAP diluted EPS is expected to be between $6.69 and $8.27 (midpoint of $7.48).

Additionally, for the fiscal year 2025 (ending June 30, 2025), revenue is projected to be between USD 26 billion and 30 billion (midpoint of USD 28 billion). A report from MoneyDJ further cite sources, indicating that Supermicro’s fiscal year 2025 revenue is expected to reach USD 23.4 billion.

According to the Q4 financial report for the fiscal year 2024 released by Supermicro, the gross margin decreased from 17.0% in the same period last year to 11.2%, the lowest since the company started reporting quarterly results in May 2007 , and below the 15.5% reported in the third quarter of the fiscal year 2024.

Per wccftech’s report, the Super Micro earnings call marked the first opportunity to engage with a company integrated into NVIDIA’s AI ecosystem. Consequently, the company’s management faced numerous questions about potential GPU delays impacting its financial performance.

In response, Charles Liang, Supermicro’s Chief Executive Officer and Chairman of the Board, acknowledged that his company had heard about potential delays from NVIDIA.

However, he emphasized that Supermicro considers such delays as a normal possibility, noting that technology companies often experience slight advancements or postponements in their schedules.

In this instance, it seemed to Liang that NVIDIA had pushed out its timeline slightly, which he stated would not impact Supermicro’s ability to offer its customers new solutions like the H200 cooling system, given their extensive customer base. He concluded by saying that the overall impact of this delay on Supermicro should be minimal.

Read more

• [News] Foxconn Remains Unfazed Despite NVIDIA’s GB200 Delay, with Customers Aim to Secure Their Orders
• [News] Supermicro President Charles Liang Bullish on Direct Liquid Cooling, Targeting a 15% Market Share within Next Year

(Photo credit: Supermicro)