Following Saudi Arabia’s $13 billion investment, the UK government is dedicating £100 million (about $130 million) to acquire thousands of NVIDIA AI chips, aiming to establish a strong global AI foothold. Potential beneficiaries include Wistron, GIGABYTE, Asia Vital Components, and Supermicro.

Projections foresee a $150 billion AI application opportunity within 3-5 years, propelling the semiconductor market to $1 trillion by 2030. Taiwan covers the full industry value chain. Players like TSMC, Alchip, GUC, Auras, Asia Vital Components, SUNON, EMC, Unimicron, Delta, and Lite-On are poised to gain.

Reports suggest the UK is in advanced talks with NVIDIA for up to 5,000 GPU chips, but models remain undisclosed. The UK government recently engaged with chip giants NVIDIA, Supermicro, Intel, and others through the UK Research and Innovation (UKRI) to swiftly acquire necessary resources for Prime Minister Sunak’s AI development initiative. Critics question the adequacy of the £100 million investment in NVIDIA chips, urging Chancellor Jeremy Hunt to allocate more funds to support the AI project.

NVIDIA’s high-performance GPU chips have gained widespread use in AI fields. Notably, the AI chatbot ChatGPT relies heavily on NVIDIA chips to meet substantial computational demands. The latest iteration of AI language model, GPT-4, requires a whopping 25,000 NVIDIA chips for training. Consequently, experts contend that the quantity of chips procured by the UK government is notably insufficient.

Of the UK’s £1 billion investment in supercomputing and AI, £900 million is for traditional supercomputers, leaving £50 million for AI chip procurement. The budget recently increased from £70 million to £100 million due to global chip demand.

Saudi Arabia and the UAE also ordered thousands of NVIDIA AI chips, and Saudi Arabia’s order includes at least 3,000 of the latest H100 chips. Prime Minister Sunak’s AI initiative begins next summer, aiming for a UK AI chatbot like ChatGPT and AI tools for healthcare and public services.

As emerging AI applications proliferate, countries are actively competing in the race to bolster AI data centers, turning the acquisition of AI-related chips into an alternative arms race. Compal said, “An anticipate significant growth in the AI server sector in 2024, primarily within hyperscale data centers, with a focus on European expansion in the first half of the year and a shift toward the US market in the latter half.”

According to a report by Taiwan’s Commercial Times, JPMorgan’s latest analysis reveals that AI demand will remain robust in the second half of the year. Encouragingly, TSMC’s CoWoS capacity expansion progress is set to exceed expectations, with production capacity projected to reach 28,000 to 30,000 wafers per month by the end of next year.

The trajectory of CoWoS capacity expansion is anticipated to accelerate notably in the latter half of 2024. This trend isn’t limited to TSMC alone; other players outside the TSMC are also actively expanding their CoWoS-like production capabilities to meet the soaring demands of AI applications.

Gokul Hariharan, Head of Research for JPMorgan Taiwan, highlighted that industry surveys indicate strong and unabated AI demand in the latter half of the year. Shortages amounting to 20% to 30% are observed with CoWoS capacity being a key bottleneck and high-bandwidth memory (HBM) also facing supply shortages.

JPMorgan’s estimates indicate that Nvidia will account for 60% of the overall CoWoS demand in 2023. TSMC is expected to produce around 1.8 to 1.9 million sets of H100 chips, followed by significant demand from Broadcom, AWS’ Inferentia chips, and Xilinx. Looking ahead to 2024, TSMC’s continuous capacity expansion is projected to supply Nvidia with approximately 4.1 to 4.2 million sets of H100 chips.

Apart from TSMC’s proactive expansion of CoWoS capacity, Hariharan predicts that other assembly and test facilities are also accelerating their expansion of CoWoS-like capacities.

For instance, UMC is preparing to have a monthly capacity of 5,000 to 6,000 wafers for the interposer layer by the latter half of 2024. Amkor is expected to provide a certain capacity for chip-on-wafer stacking technology, and ASE Group will offer chip-on-substrate bonding capacity. However, these additional capacities might face challenges in ramping up production for the latest products like H100, potentially focusing more on older-generation products like A100 and A800.

(Photo credit: TSMC)

According to a report by Taiwan’s Economic Daily, Foxconn Group has achieved another triumph in its AI endeavors. The company has secured orders for over 50% of NVIDIA’s HGX GPU base boards, marking the first instance of such an achievement. Adding to this success, Foxconn had previously acquired an order for another NVIDIA DGX GPU base board, solidifying its pivotal role in NVIDIA’s two most crucial AI chip base board orders.

The report highlights that in terms of supply chain source, Foxconn Group stands as the exclusive provider of NVIDIA’s AI chip modules (GPU Modules). As for NVIDIA’s AI motherboards, the suppliers encompass Foxconn, Quanta, Inventec, and Super Micro.

Industry experts analyze that DGX and HGX are currently NVIDIA’s two most essential AI servers, and Foxconn Group has undertaken the monumental task of fulfilling the large order for NVIDIA’s AI chipboards through its subsidiary, Foxconn Industrial Internet (FII). Having previously secured orders for NVIDIA’s DGX base boards, Foxconn Group has now garnered additional orders from FII for the HGX base boards. This expanded supply constitutes more than half of the total, solidifying Foxconn Group’s role as a primary supplier for NVIDIA’s two critical AI chip base board orders.

Furthermore, Foxconn’s involvement doesn’t end with AI chip modules, base boards, and motherboards. The company’s engagement extends downstream to servers and server cabinets, creating a vertically integrated approach that covers the entire AI ecosystem.

(Photo credit: Nvidia)

AI Chips and High-Performance Computing (HPC) have been continuously shaking up the entire supply chain, with CoWoS packaging technology being the latest area to experience the tremors.

In the previous piece, “HBM and 2.5D Packaging: the Essential Backbone Behind AI Server,” we discovered that the leading AI chip players, Nvidia and AMD, have been dedicated users of TSMC’s CoWoS technology. Much of the groundbreaking tech used in their flagship product series – such as Nvidia’s A100 and H100, and AMD’s Instinct MI250X and MI300 – have their roots in TSMC’s CoWoS tech.

However, with AI’s exponential growth, chip demand from not just Nvidia and AMD has skyrocketed, but other giants like Google and Amazon are also catching up in the AI field, bringing an onslaught of chip demand. The surge of orders is already testing the limits of TSMC’s CoWoS capacity. While TSMC is planning to increase its production in the latter half of 2023, there’s a snag – the lead time of the packaging equipment is proving to be a bottleneck, severely curtailing the pace of this necessary capacity expansion.

Nvidia Shakes the foundation of the CoWoS Supply Chain

In these times of booming demand, maintaining a stable supply is viewed as the primary goal for chipmakers, including Nvidia. While TSMC is struggling to keep up with customer needs, other chipmakers are starting to tweak their outsourcing strategies, moving towards a more diversified supply chain model. This shift is now opening opportunities for other foundries and OSATs.

Interestingly, in this reshuffling of the supply chain, UMC (United Microelectronics Corporation) is reportedly becoming one of Nvidia’s key partners in the interposer sector for the first time, with plans for capacity expansion on the horizon.

From a technical viewpoint, interposer has always been the cornerstone of TSMC’s CoWoS process and technology progression. As the interposer area enlarges, it allows for more memory stack particles and core components to be integrated. This is crucial for increasingly complex multi-chip designs, underscoring Nvidia’s intention to support UMC as a backup resource to safeguard supply continuity.

Meanwhile, as Nvidia secures production capacity, it is observed that the two leading OSAT companies, Amkor and SPIL (as part of ASE), are establishing themselves in the Chip-on-Wafer (CoW) and Wafer-on-Substrate (WoS) processes.

The ASE Group is no stranger to the 2.5D packaging arena. It unveiled its proprietary 2.5D packaging tech as early as 2017, a technology capable of integrating core computational elements and High Bandwidth Memory (HBM) onto the silicon interposer. This approach was once utilized in AMD’s MI200 series server GPU. Also under the ASE Group umbrella, SPIL boasts unique Fan-Out Embedded Bridge (FO-EB) technology. Bypassing silicon interposers, the platform leverages silicon bridges and redistribution layers (RDL) for integration, which provides ASE another competitive edge.

Could Samsung’s Turnkey Service Break New Ground?

In the shifting landscape of the supply chain, the Samsung Device Solutions division’s turnkey service, spanning from foundry operations to Advanced Package (AVP), stands out as an emerging player that can’t be ignored.

After its 2018 split, Samsung Foundry started taking orders beyond System LSI for business stability. In 2023, the AVP department, initially serving Samsung’s memory and foundry businesses, has also expanded its reach to external clients.

Our research indicates that Samsung’s AVP division is making aggressive strides into the AI field. Currently in active talks with key customers in the U.S. and China, Samsung is positioning its foundry-to-packaging turnkey solutions and standalone advanced packaging processes as viable, mature options.

In terms of technology roadmap, Samsung has invested significantly in 2.5D packaging R&D. Mirroring TSMC, the company launched two 2.5D packaging technologies in 2021: the I-Cube4, capable of integrating four HBM stacks and one core component onto a silicon interposer, and the H-Cube, designed to extend packaging area by integrating HDI PCB beneath the ABF substrate, primarily for designs incorporating six or more HBM stack particles.

Besides, recognizing Japan’s dominance in packaging materials and technologies, Samsung recently launched a R&D center there to swiftly upscale its AVP business.

Given all these circumstances, it seems to be only a matter of time before Samsung carves out its own significant share in the AI chip market. Despite TSMC’s industry dominance and pivotal role in AI chip advancements, the rising demand for advanced packaging is set to undeniably reshape supply chain dynamics and the future of the semiconductor industry.

(Source: Nvidia)

High Bandwidth Memory (HBM) is emerging as the preferred solution for overcoming memory transfer speed restrictions due to the bandwidth limitations of DDR SDRAM in high-speed computation. HBM is recognized for its revolutionary transmission efficiency and plays a pivotal role in allowing core computational components to operate at their maximum capacity. Top-tier AI server GPUs have set a new industry standard by primarily using HBM. TrendForce forecasts that global demand for HBM will experience almost 60% growth annually in 2023, reaching 290 million GB, with a further 30% growth in 2024.

TrendForce’s forecast for 2025, taking into account five large-scale AIGC products equivalent to ChatGPT, 25 mid-size AIGC products from Midjourney, and 80 small AIGC products, the minimum computing resources required globally could range from 145,600 to 233,700 Nvidia A100 GPUs. Emerging technologies such as supercomputers, 8K video streaming, and AR/VR, among others, are expected to simultaneously increase the workload on cloud computing systems due to escalating demands for high-speed computing.

HBM is unequivocally a superior solution for building high-speed computing platforms, thanks to its higher bandwidth and lower energy consumption compared to DDR SDRAM. This distinction is clear when comparing DDR4 SDRAM and DDR5 SDRAM, released in 2014 and 2020 respectively, whose bandwidths only differed by a factor of two. Regardless of whether DDR5 or the future DDR6 is used, the quest for higher transmission performance will inevitably lead to an increase in power consumption, which could potentially affect system performance adversely. Taking HBM3 and DDR5 as examples, the former’s bandwidth is 15 times that of the latter and can be further enhanced by adding more stacked chips. Furthermore, HBM can replace a portion of GDDR SDRAM or DDR SDRAM, thus managing power consumption more effectively.

TrendForce concludes that the current driving force behind the increasing demand is AI servers equipped with Nvidia A100, H100, AMD MI300, and large CSPs such as Google and AWS, which are developing their own ASICs. It is estimated that the shipment volume of AI servers, including those equipped with GPUs, FPGAs, and ASICs, will reach nearly 1.2 million units in 2023, marking an annual growth rate of almost 38%. TrendForce also anticipates a concurrent surge in the shipment volume of AI chips, with growth potentially exceeding 50%.