According to the news from Chinatimes, Asus, a prominent technology company, has announced on the 30th of this month the release of AI servers equipped with NVIDIA’s L40S GPUs. These servers are now available for order. The L40S GPU was introduced by NVIDIA in August to address the shortage of H100 and A100 GPUs. Remarkably, Asus has swiftly responded to this situation by unveiling AI server products within a span of less than two weeks, showcasing their optimism in the imminent surge of AI applications and their eagerness to seize the opportunity.

Solid AI Capabilities of Asus Group

Apart from being among the first manufacturers to introduce the NVIDIA OVX server system, Asus has leveraged resources from its subsidiaries, such as TaiSmart and Asus Cloud, to establish a formidable AI infrastructure. This not only involves in-house innovation like the Large Language Model (LLM) technology but also extends to providing AI computing power and enterprise-level generative AI applications. These strengths position Asus as one of the few all-encompassing providers of generative AI solutions.

Projected Surge in Server Business

Regarding server business performance, Asus envisions a yearly compounded growth rate of at least 40% until 2027, with a goal of achieving a fivefold growth over five years. In particular, the data center server business catering primarily to Cloud Service Providers (CSPs) anticipates a tenfold growth within the same timeframe, driven by the adoption of AI server products.

Asus CEO recently emphasized that Asus’s foray into AI server development was prompt and involved collaboration with NVIDIA from the outset. While the product lineup might be more streamlined compared to other OEM/ODM manufacturers, Asus had secured numerous GPU orders ahead of the AI server demand surge. The company is optimistic about the shipping momentum and order visibility for the new generation of AI servers in the latter half of the year.

Embracing NVIDIA’s Versatile L40S GPU

The NVIDIA L40S GPU, built on the Ada Lovelace architecture, stands out as one of the most powerful general-purpose GPUs in data centers. It offers groundbreaking multi-workload computations for large language model inference, training, graphics, and image processing. Not only does it facilitate rapid hardware solution deployment, but it also holds significance due to the current scarcity of higher-tier H100 and A100 GPUs, which have reached allocation stages. Consequently, businesses seeking to repurpose idle data centers are anticipated to shift their focus toward AI servers featuring the L40S GPU.

Asus’s newly introduced L40S GPU servers include the ESC8000-E11/ESC4000-E11 models with built-in Intel Xeon processors, as well as the ESC8000A-E12/ESC4000A-E12 models utilizing AMD EPYC processors. These servers can be configured with up to 4 or a maximum of 8 NVIDIA L40S GPUs. This configuration assists enterprises in enhancing training, fine-tuning, and inference workloads, facilitating AI model creation. It also establishes Asus’s platforms as the preferred choice for multi-modal generative AI applications.

(Source: https://www.chinatimes.com/newspapers/20230831000158-260202?chdtv)

According to a report from Taiwan’s TechNews, NVIDIA has delivered impressive results in its latest financial report, coupled with an optimistic outlook for its financial projections. This demonstrates that the demand for AI remains robust for the coming quarters. Currently, NVIDIA’s H100 and A100 chips both utilize TSMC’s CoWoS advanced packaging technology, making TSMC’s production capacity a crucial factor.

Examining the core GPU market, NVIDIA holds a dominant market share of 90%, while AMD accounts for about 10%. While other companies might adopt Google’s TPU or develop customized chips, they currently lack significant operational cost advantages.

In the short term, the shortage of CoWoS has led to tight chip supplies. However, according to a recent report by Morgan Stanley Securities, NVIDIA believes that TSMC’s CoWoS capacity won’t restrict shipments of the next quarter’s H100 GPUs. The company anticipates an increase in supply for each quarter next year. Simultaneously, TSMC is raising CoWoS prices by 20% for rush orders, indicating that the anticipated CoWoS bottleneck might alleviate.

According to industry sources, NVIDIA is actively diversifying its CoWoS supply chain away from TSMC. UMC, ASE, Amkor, and SPIL are significant players in this effort. Currently, UMC is expanding its interposer production capacity, aiming to double its capacity to relieve the tight CoWoS supply situation.

According to Morgan Stanley Securities, TSMC’s monthly CoWoS capacity this year is around 11,000 wafers, projected to reach 25,000 wafers by the end of next year. Non-TSMC CoWoS supply chain’s monthly capacity can reach 3,000 wafers, with a planned increase to 5,000 wafers by the end of next year.

(Photo credit: TSMC)

NVIDIA Beats Expectations with Q2 Financial Results and Optimistic Q3 Outlook, But Overall Semiconductor Short-Term Prospects Remain Weak, According to Taiwan’s Central News Agency.

While the semiconductor industry remains subdued, NVIDIA stands out with robust operational performance and a positive outlook. The company reported Q2 revenue of $13.51 billion, an 88% increase from the previous quarter and double the figure from the same period last year. Net income reached $6.19 billion, translating to $2.48 per share. NVIDIA anticipates Q3 revenue to further reach around $16 billion, marking a 170% YoY increase.

According to research firm TrendForce, NVIDIA’s rapid data center business growth is the primary driver. In Q4 of the fiscal year 2022, data center revenue accounted for about 42.7% of the total, surpassing gaming. In Q1 of FY 2023, it exceeded 45%, and by Q2 of FY 2024, data center revenue reached $10.32 billion, a 141% increase from the previous quarter and a 171% YoY increase, making up more than 76% of total revenue.

TrendForce notes that AI server solutions are pivotal in propelling NVIDIA’s data center growth, including AI accelerator GPUs and AI server reference architecture like HGX.

Arisa Liu, a researcher and director at Taiwan Industry Economics Services, mentioned that NVIDIA’s outstanding performance underscores its solid leadership in the AI market. She emphasized that customer demand for AI-related solutions is consistently on the rise.

Liu also mentioned that NVIDIA’s supply chain is expected to benefit in tandem. Orders for TSMC’s 7nm, 4nm, and 3nm advanced processes might increase. Advanced packaging technologies like CoWoS are expected to remain in high demand. In addition, orders for silicon intellectual property, high-speed transmission components, power supply, PCBs, chassis, and server OEMs are likely to see growth.

However, Liu indicated that due to the relatively low share of the AI market, it cannot fully offset the impact of sluggish demand in major application markets such as computers, smartphones, and consumer electronics. As a result, the short-term semiconductor market conditions are expected to remain weak.

(Photo credit: NVIDIA)

In the face of adversities within the autonomous vehicle market, car manufacturers are not hitting the brakes. Rather, they’re zeroing in, adopting more focused and streamlined strategies, deeply rooted in core technologies.

Eager to expedite the mass-scale rollout of Robotaxis, Tesla recently announced an acceleration in the development of their Dojo supercomputer. They are now committing an investment of $1 billion and set to have 100,000 NVIDIA A100 GPUs ready by early 2024, potentially placing them among the top five global computing powerhouses.

While Tesla already boasts a supercomputer built on NVIDIA GPUs, they’re still passionate about crafting a highly efficient one in-house. This move signifies that computational capability is becoming an essential arsenal for automakers, reflecting the importance of mastering R&D in this regard.

HPC Fosters Collaboration in the Car Ecosystem

According to forecasts from TrendForce, the global high-performance computing(HPC) market could touch $42.6 billion by 2023, further expanding to $56.8 billion by 2027 with an annual growth rate of over 7%. And it is highly believed that the automotive sector is anticipated to be the primary force propelling this growth.

Feeling the heat of industry upgrades, major automakers like BMW, Continental, General Motors, and Toyota aren’t just investing in high-performance computing systems; they’re also forging deep ties with ecosystem partners, enhancing cloud, edge, chip design, and manufacturing technologies.

For example, BMW, who’s currently joining forces with EcoDataCenter, is currently seeking to extend its high-performance computing footprint, aiming to elevate their autonomous driving and driver-assist systems.

On another front, Continental, the leading tier-1 supplier, is betting on its cross-domain integration and scalable CAEdge (Car Edge framework). Set to debut in the first half of 2023, this solution for smart cockpits offers automakers a much more flexible development environment.

In-house Tech Driving Towards Level 3 and Beyond

To successfully roll out autonomous driving on a grand scale, three pillars are paramount: extensive real-world data, neural network training, and in-vehicle hardware/software. None can be overlooked, thereby prompting many automakers and Tier 1 enterprises to double down on their tech blueprints.

Tesla has already made significant strides in various related products. Beyond their supercomputer plan, their repertoire includes the D1 chip, Full Self-Driving (FSD) computation, multi-camera neural networks, and automated tagging, with inter-platform data serving as the backbone for their supercomputer’s operations.

In a similar vein, General Motors’ subsidiary, Cruise, while being mindful of cost considerations, is gradually phasing out NVIDIA GPUs, opting instead to develop custom ASIC chips to power its vehicles.

Another front-runner, Valeo, unveiled their Scala 3 in the first half of 2023, nudging LiDAR technology closer to Level 3, and laying a foundation for robotaxi(Level 4) deployment.

All this paints a picture – even with a subdued auto market, car manufacturers’ commitment to autonomous tech R&D hasn’t waned. In the long run, those who steadfastly stick to their tech strategies and nimbly adjust to market fluctuations are poised to lead the next market resurgence, becoming beacons in the industry.

For more information on reports and market data from TrendForce’s Department of Semiconductor Research, please click here, or email Ms. Latte Chung from the Sales Department at lattechung@trendforce.com

(Photo credit: Tesla)

NVIDIA’s latest financial report for FY2Q24 reveals that its data center business reached US$10.32 billion—a QoQ growth of 141% and YoY increase of 171%. The company remains optimistic about its future growth. TrendForce believes that the primary driver behind NVIDIA’s robust revenue growth stems from its data center’s AI server-related solutions. Key products include AI-accelerated GPUs and AI server HGX reference architecture, which serve as the foundational AI infrastructure for large data centers.

TrendForce further anticipates that NVIDIA will integrate its software and hardware resources. Utilizing a refined approach, NVIDIA will align its high-end, mid-tier, and entry-level GPU AI accelerator chips with various ODMs and OEMs, establishing a collaborative system certification model. Beyond accelerating the deployment of CSP cloud AI server infrastructures, NVIDIA is also partnering with entities like VMware on solutions including the Private AI Foundation. This strategy extends NVIDIA’s reach into the edge enterprise AI server market, underpinning steady growth in its data center business for the next two years.

NVIDIA’s data center business surpasses 76% market share due to strong demand for cloud AI

In recent years, NVIDIA has been actively expanding its data center business. In FY4Q22, data center revenue accounted for approximately 42.7%, trailing its gaming segment by about 2 percentage points. However, by FY1Q23, data center business surpassed gaming—accounting for over 45% of revenue. Starting in 2023, with major CSPs heavily investing in ChatBOTS and various AI services for public cloud infrastructures, NVIDIA reaped significant benefits. By FY2Q24, data center revenue share skyrocketed to over 76%.

NVIDIA targets both Cloud and Edge Data Center AI markets

TrendForce observes and forecasts a shift in NVIDIA’s approach to high-end GPU products in 2H23. While the company has primarily focused on top-tier AI servers equipped with the A100 and H100, given positive market demand, NVIDIA is likely to prioritize the higher-priced H100 to effectively boost its data-center-related revenue growth.

NVIDIA is currently emphasizing the L40s as their flagship product for mid-tier GPUs, meaning several strategic implications: Firstly, the high-end H100 series is constrained by the limited production capacity of current CoWoS and HBM technologies. In contrast, the L40s primarily utilizes GDDR memory. Without the need for CoWos packaging, it can be rapidly introduced to the mid-tier AI server market, filling the gap left by the A100 PCle interface in meeting the needs of enterprise customers.

Secondly, the L40s also target enterprise customers who don’t require large parameter models like ChatGPT. Instead, it focuses on more compact AI training applications in various specialized fields, with parameter counts ranging from tens of billions to under a hundred billion. They can also address edge AI inference or image analysis tasks. Additionally, in light of potential geopolitical issues that might disrupt the supply of the high-end GPU H series for Chinese customers, the L40s can serve as an alternative. As for lower-tier GPUs, NVIDIA highlights the L4 or T4 series, which are designed for real-time AI inference or image analysis in edge AI servers. These GPUs underscore affordability while maintaining a high-cost-performance ratio.

HGX and MGX AI server reference architectures are set to be NVIDIA’s main weapons for AI solutions in 2H23

TrendForce notes that recently, NVIDIA has not only refined its product positioning for its core AI chip GPU but has also actively promoted its HGX and MGX solutions. Although this approach isn’t new in the server industry, NVIDIA has the opportunity to solidify its leading position with this strategy. The key is NVIDIA’s absolute leadership stemming from its extensive integration of its GPU and CUDA platform—establishing a comprehensive AI ecosystem. As a result, NVIDIA has considerable negotiating power with existing server supply chains. Consequently, ODMs like Inventec, Quanta, FII, Wistron, and Wiwynn, as well as brands such as Dell, Supermicro, and Gigabyte, are encouraged to follow NVIDIA’s HGX or MGX reference designs. However, they must undergo NVIDIA’s hardware and software certification process for these AI server reference architectures. Leveraging this, NVIDIA can bundle and offer integrated solutions like its Arm CPU Grace, NPU, and AI Cloud Foundation.

It’s worth noting that for ODMs or OEMs, given that NVIDIA is expected to make significant achievements in the AI server market for CSPs from 2023 to 2024, there will likely be a boost in overall shipment volume and revenue growth of AI servers. However, with NVIDIA’s strategic introduction of standardized AI server architectures like HGX or MGX, the core product architecture for AI servers among ODMs and others will become more homogenized. This will intensify the competition among them as they vie for orders from CSPs. Furthermore, it’s been observed that large CSPs such as Google and AWS are leaning toward adopting in-house ASIC AI accelerator chips in the future, meaning there’s a potential threat to a portion of NVIDIA’s GPU market. This is likely one of the reasons NVIDIA continues to roll out GPUs with varied positioning and comprehensive solutions. They aim to further expand their AI business aggressively to Tier-2 data centers (like CoreWeave) and edge enterprise clients.