The Executive Yuan and TSMC have reportedly reached a consensus on the investment project for the new advanced packaging plant at the TSMC Science Park in Chiayi. According to a report from Economic Daily News, six new plant sites will be allocated to TSMC in the Science Park, two more than originally anticipated, with a total investment exceeding NTD 500 billion. The expansion is expected to increase CoWoS advanced packaging capacity and to be announced to the public in early April.

TSMC has refrained from commenting on the matter. Regarding the news, the Executive Yuan actively coordinated with TSMC for the establishment of the advanced packaging plant in the Chiayi Science Park located in Taibao. The related environmental assessments and water and electricity facilities have been processed, with construction expected to commence in April, indirectly confirming the rumors.

As per sources cited by the report, Chiayi Science Park is poised to become a new hub for TSMC’s advanced packaging capacity. Among the six new scheduled plants, construction will begin on two this year, aligning with the Executive Yuan’s statement of construction commencement in April.

TSMC’s extensive expansion is primarily driven by the high demand for advanced packaging. For instance,  in the case of the NVIDIA H100, after integrating components via CoWoS, each wafer yields approximately 28 chips. However, for the upcoming B100, with increased volume and integration, the yield per wafer drops to just 16 chips.

On the other hand, TSMC’s advanced processes, per a previous report from Commercial Times, remained fully utilized, with capacity utilization exceeding 90% in February, driven by sustained AI demand. The same report also noted that NVIDIA’s orders to TSMC are robust, pushing TSMC’s 3 and 4-nanometer production capacity to nearly full utilization.

As each new generation of NVIDIA’s AI chips integrates CoWoS, chip output is halved, yet demand for AI servers continues to soar. With terminal demand skyrocketing while chip output dwindles, there’s a “cliff-like gap” in CoWoS advanced packaging capacity. TSMC must ramp up CoWoS production swiftly to ensure uninterrupted customer supply.

Read more

• [News] SK Hynix Reportedly Invests USD 1 Billion in Advanced Packaging to Strengthen HBM Chip Leadership
• [News] TSMC Reportedly Doubles CoWoS Capacity while Amkor, ASE also Enter Advanced Packaging for AI

(Photo credit: TSMC)

“It is not the shortage of AI chips, it is the shortage of our CoWoS capacity,” replied TSMC Chairman Mark Liu during an interview in September last year, propelling this technology that TSMC had quietly cultivated for over a decade into a global spotlight.

As per a report from TechNews, the hardware demand sparked by generative AI has also led to “advanced packaging” becoming not only a hot keyword pursued by global investors but also a prominent feature of the semiconductor industry. From foundries and memory manufacturers to OASTs, all are actively involved in the research and capacity expansion of advanced packaging technologies.

TSMC, the leading force in the advanced packaging market, has repeatedly emphasized its efforts to expand capacity during its earnings call, including capacity expansions in Zhunan and Hsinchu, and even the possibility of constructing advanced packaging facilities in Chiayi.

Intel’s strategic moves also underscore its emphasis on the development of advanced packaging. Intel’s new plant completed in Penang, Malaysia in 2023 is aimed at establishing advanced packaging capacity.

Leading packaging and testing company, ASE Technology Holding, has also actively participated in the competition for advanced packaging. Apart from its subsidiary, Siliconware Precision Industries (SPIL), which is already a supplier for the backend packaging of CoWoS, ASE Technology Holding is also expanding advanced packaging capacity at its facility in Kaohsiung.

Memory manufacturers are also aggressively ramping up their advanced packaging capacity. SK Hynix, which exclusively supplies HBM for NVIDIA AI chips, recently announced plans to invest USD 1 billion in the development of advanced packaging. They view advanced packaging as the “future focus of semiconductor development for the next 50 years.

Advanced Packaging: Over a Decade of Development

In fact, advanced packaging is not a new concept. Tracing the history of packaging technology, the year 2000 undoubtedly marked a turning point. From this year onwards, packaging technology shifted from traditional wire bonding and flip-chip methods to “wafer-level packaging,” where most or all packaging and testing processes are conducted on the wafer itself.

The 2.5D packaging, which gained significant attention after 2023, actually emerged as early as 2010. However, due to cost concerns, the number of manufacturers adopting this technology was relatively limited, with a focus on high-performance computing chips.

Chiang Shang-yi, the Chief Strategy Officer of Foxconn Semiconductor, recalled the initial lack of interest in CoWoS technology, which even led to him being regarded as a “joke” within the company( TSMC) for proposing advanced packaging. He also revealed that the first company willing to adopt the costly CoWoS technology was actually Huawei.

▲ Semiconductor Packaging Technology Evolution compiled by McKinsey, Accelerated Technological Evolution after 2000 (Source: McKinsey)

Compared to 2D packaging technology, 2.5D packaging involves placing an intermediate layer between the chip and the IC substrate and stacking different chips in parallel. TSMC’s CoWoS has become synonymous with 2.5D packaging, where a silicon interposer layer is inserted between the chip and the SiP substrate, and metal layers are connected using Through-Silicon Vias (TSVs) to overcome the density limitations of SiP substrates, which previously restricted the number of chips.

Despite TSMC’s dominance, Intel, with its extensive technical expertise in CPU packaging, cannot be underestimated. In the 2.5D packaging battlefield, Intel employs EMIB technology as its strategy. Unlike CoWoS, EMIB does not utilize a silicon interposer layer.

Instead, its key feature lies in the “Silicon Bridge,” buried within the packaging substrate, which connects the bare dies. Intel believes that EMIB offers cost advantages compared to solutions using large silicon interposer layers.

In recent years, Samsung, which has been actively cultivating the semiconductor foundry market, has also ventured into the 2.5D packaging arena. Their proprietary I-Cube technology has traditionally targeted applications in High-Performance Computing (HPC) chips. When Samsung introduced I-Cube4 in 2021, it emphasized the integration of multiple logic dies and HBM placed on a silicon interposer layer, enabling heterogeneous integration into a single chip.

As Moore’s Law approaches its limits and the massive computational demands triggered by generative AI continue to surge, coupled with the trend towards lighter, thinner, and smaller end products, chips are inevitably evolving towards more transistors, greater computational power, and lower power consumption performance.

Therefore, the transition of packaging technology from 2.5D to 3D is undoubtedly an inevitable development.

The difference between 3D and 2.5D packaging lies in the stacking method. In 2.5D packaging, chips are stacked parallelly on an intermediate layer, while in 3D packaging, chips are stacked vertically in a three-dimensional manner.

The advantage of 3D packaging lies in its ability to create more space for transistors within a chip through stacking, shorten the distance between different bare dies significantly, enhance transmission efficiency, and reduce power consumption during transmission.

TSMC, Intel, and Samsung Racing for 3D Packaging Technology

TSMC’s positioning in 3D IC technology is undeniable. Its SoIC technology adopts the wafer-to-wafer bonding technique. SoIC integrates homogeneous and heterogeneous small dies into a single chip, with smaller dimensions and a thinner profile. It can be integrated into 2.5D CoWoS or InFO. From an external perspective, SoIC resembles a universal SoC chip but integrates various functions heterogeneously.

Intel’s layout in 3D packaging revolves around its 3D Foveros technology. Structurally, the bottom layer comprises a packaging substrate, with a bottom wafer placed on top serving as an intermediate layer. Within this intermediate layer, numerous TSVs (Through-Silicon Vias) are present, facilitating connections between the upper chips, modules, and other parts of the system to achieve transmission purposes.

Samsung’s X-Cube 3D packaging technology utilizes TSV processes. Currently, Samsung’s X-Cube test chips can stack the SRAM layer on top of the logic layer, interconnected via TSVs, employing its 7nm EUV process technology.

TSMC’s Comprehensive Ecosystem Strategy

Currently dominating the advanced packaging market, thanks to its acquisition of large contracts for manufacturing NVIDIA AI chips, TSMC is not only continuing to develop more advanced packaging technologies but is also actively promoting its 3D Fabric platform.

In addition to incorporating the three key packaging technologies CoWoS, InFo, and SoIC, this platform has expanded into an industry alliance. It includes participation from EDA, IP, DCA/VCA, memory, packaging and testing suppliers, as well as substrate and testing vendors. The goal is to create a complete 3D Fabric ecosystem, strengthen innovation, and enhance customer adoption willingness.

This alliance has attracted active participation from heavyweight players in the semiconductor upstream supply chain. Even companies traditionally seen as competitors in the packaging and testing sector, such as Amkor, ASE Technology Holding, and Siliconware Precision Industries (SPIL), are members. The comprehensive supply chain has become a significant advantage for TSMC in providing advanced packaging contract manufacturing services.

▲ TSMC’s 3D Fabric Alliance members, including major players from EDA to packaging and testing companies. (Image Source: TSMC)

In comparison, Intel, despite its robust technological expertise developed over many years and its proposition to independently provide wafer manufacturing or testing services, faces a disadvantage in expanding its market share in advanced packaging due to its lack of experience in the foundry market.

On the other hand, Samsung, compared to TSMC, is constrained by its yield issues in advanced processes. This limitation leads IC design companies to prioritize foundries with more stable yields when considering outsourcing comprehensive manufacturing services.

Read more

• [News] SK Hynix Reportedly Invests USD 1 Billion in Advanced Packaging to Strengthen HBM Chip Leadership
• [News] TSMC Reportedly Doubles CoWoS Capacity while Amkor, ASE also Enter Advanced Packaging for AI

(Photo credit: TSMC)

With numerous cloud computing companies and large-scale AI model manufacturers investing heavily in AI computing infrastructure, the demand for AI processors is rapidly increasing. As per a report from IJIWEI, the demand for HBM (High Bandwidth Memory), a key component among them, has been on the rise as well.

Amid the opportunity brought about by the surge in demand for computing power, which has in turn created a wave of opportunities for storage capabilities, when looking at the entire HBM industry chain, the number of China’s local companies which are able to enter the field is limited.

Faced with significant technological challenges but vast prospects, whether from the perspective of independent controllability or market competition, it is imperative to accelerate the pace of catching up.

HBM Demand Grows Against the Trend, Dominated by Three Giants

The first TSV HBM product debuted in 2014, but it wasn’t until after the release of ChatGPT in 2023 that the robust demand for AI servers drove rapid iterations of HBM technology in the order of HBM1, HBM2, HBM2e, HBM3, and HBM3e.

The fourth-generation HBM3 has been mass-produced and applied, with significant improvements in bandwidth, stack height, capacity, I/O speed, and more compared to the first generation. Currently, only three storage giants—SK Hynix, Samsung Electronics, and Micron—are capable of mass-producing HBM.

According to a previous TrendForce press release, the three major original HBM manufacturers held market shares as follows in 2023: SK Hynix and Samsung were both around 46-49%, while Micron stood at roughly 4-6%.

In 2023, the primary applications in the market were HBM2, HBM2e, and HBM3, with the penetration rate of HBM3 increasing in the latter half of the year due to the push from NVIDIA’s H100 and AMD’s MI300.

According to TrendForce’s report, SK Hynix led the way with its HBM3e validation in the first quarter, closely followed by Micron, which plans to start distributing its HBM3e products toward the end of the first quarter, in alignment with NVIDIA’s planned H200 deployment by the end of the second quarter.

Samsung, slightly behind in sample submissions, is expected to complete its HBM3e validation by the end of the first quarter, with shipments rolling out in the second quarter.

Driven by market demand, major players such as SK Hynix, Samsung, and Micron Technology are increasing their efforts to expand production capacity. SK Hynix revealed in February that all its HBM products had been fully allocated for the year, prompting preparations for 2025 to maintain market leadership.

Reportedly, Samsung, aiming to compete in the 2024 HBM market, plans to increase the maximum production capacity to 150,000 to 170,000 units per month before the end of the fourth quarter of this year. Previously, Samsung also invested KRW 10.5 billion to acquire Samsung Display’s factory and equipment in Cheonan, South Korea, with the aim of expanding HBM production capacity.

Micron Technology CEO Sanjay Mehrotra recently revealed that Micron’s HBM production capacity for 2024 is expected to be fully allocated.

Although the three major HBM suppliers continue to focus on iterating HBM3e, there is still room for improvement in single-die DRAM and stacking layers. However, the development of HBM4 has been put on the agenda.

Trendforce previously predicted that HBM4 will mark the first use of a 12nm process wafer for its bottommost logic die (base die), to be supplied by foundries. This advancement signifies a collaborative effort between foundries and memory suppliers for each HBM product, reflecting the evolving landscape of high-speed memory technology.

Continuous Surge in HBM Demand and Prices, Local Supply Chains in China Catching Up

In the face of a vast market opportunity, aside from the continuous efforts of the three giants to ramp up research and production, some second and third-tier Chinese DRAM manufacturers have also entered the HBM race. With the improvement in the level of locally produced AI processors, the demand for independent HBM supply chains in China has become increasingly urgent.

Top global manufacturers operate DRAM processes at the 1alpha and 1beta levels, while China’s DRAM processes operate at the 25-17nm level. China’s DRAM processes are approaching those overseas, and there are advanced packaging technology resources and GPU customer resources locally, indicating a strong demand for HBM localization. In the future, local DRAM manufacturers in China are reportedly expected to break through into HBM.

It is worth noting that the research and manufacturing of HBM involve complex processes and technical challenges, including wafer-level packaging, testing technology, design compatibility, and more. CoWoS is currently the mainstream packaging solution for AI processors, and in AI chips utilizing CoWoS technology, HBM integration is also incorporated.

CoWoS and HBM involves processes such as TSV (Through-Silicon Via), bumps, microbumps, and RDL (Redistribution Layer). Among these, TSV accounts for the highest proportion of the 3D packaging cost of HBM, close to 30%.

Currently, China has only a few leading packaging companies such as JCET Group, Tongfu Microelectronics, and SJSemi that possess the technology (such as TSV through-silicon via) and equipment required to support HBM production.

However, despite these efforts, the number of Chinese companies truly involved in the HBM industry chain remains limited, with most focusing on upstream materials.
With GPU acquisition restricted, breakthroughs in China’s AI processors are urgently needed both from its own self-sufficiency perspective and in terms of market competition. Therefore, synchronized breakthroughs in HBM are also crucial from Chinese manufacturers.

Read more

• [News] Another Semiconductor Giant Makes a Move on HBM
• High-End AI Server Demand from North America’s Top Four CSPs Expected to Exceed 60% in 2024, Says TrendForce

(Photo credit: SK Hynix)

With AI demand exploding across the board, TSMC initiated a major expansion plan for CoWoS in 2023. According to MoneyDJ citing sources, TSMC reportedly initiated a new wave of orders to Taiwan-based equipment manufacturers this month. Delivery is expected in the fourth quarter of this year.

Therefore, by the end of 2024, monthly production capacity may have the chance to double from the company’s target and further exceed market estimates of 35,000 wafers, possibly reaching over 40,000 wafers. As per the same report, TSMC is making a full-scale push to expand its CoWoS production capacity, aiming for a doubling of growth by 2024, with continued expansion expected in 2025.

As per multiple sources cited by MoneyDJ, TSMC resumed ordering CoWoS equipment in April 2023, with the second and third waves of additional orders placed in June and October, respectively. Subsequently, there were sporadic additional orders.

However, this month, there is a new wave of proactive orders, scheduled for delivery in the fourth quarter. Initially, it was estimated that CoWoS monthly production capacity would reach 32,000 to 35,000 wafers by the end of 2024. Now, it is possible that it will exceed 40,000 wafers.

Regarding TSMC’s SoIC, following AMD, Apple also plans to adopt this technology, intending to use SoIC combined with Hybrid molding. Currently, it is undergoing small-scale trial production.

To meet customer demand, TSMC continues to revise its capacity plans. At the end of last year, SoIC monthly capacity was around 2,000 wafers, with a target to reach nearly 6,000 wafers by the end of this year. The monthly capacity target for 2025 is to double again to over 14,000 to 15,000 wafers.

Still, as per industry sources cited by CNA in its previous report, the demand for advanced packaging capacity for AI chips still outstrips supply. The report also revealed that NVIDIA has sought assistance from packaging and testing subcontractors outside of TSMC to augment their advanced packaging capabilities.

Amkor, among others, began gradually providing capacity support from the fourth quarter of last year, while SPIL, a subsidiary of ASE, is slated to commence supply in the first quarter of this year.

Read more

• [News] TSMC to Expand Production with New Facilities Across Taiwan in April
• [News] TSMC Reportedly Doubles CoWoS Capacity while Amkor, ASE also Enter Advanced Packaging for AI

(Photo credit: TSMC)

Last year’s AI boom propelled NVIDIA into the spotlight, yet the company finds itself at a challenging crossroads.

According to a report from TechNews, on one hand, NVIDIA dominates in high-performance computing and artificial intelligence, continuously expanding with its latest GPU products. On the other hand, global supply chain instability, rapid emergence of competitors, and uncertainties in technological innovation are exerting unprecedented pressure on NVIDIA.

NVIDIA’s stock price surged by 246% last year, driving its market value past USD 1 trillion and making it the first chip company to achieve this milestone. According to the Bloomberg Billionaires Index, NVIDIA CEO Jensen Huang’s personal wealth has soared to USD 55.7 billion.

However, despite the seemingly radiant outlook for the NVIDIA, as per a report from TechNews, it still faces uncontrollable internal and external challenges.

• Internal Concern 1: CoWoS, HBM Capacity Bottlenecks

The most apparent issue lies in capacity constraints.

Currently, NVIDIA’s A100 and H100 GPUs are manufactured using TSMC’s CoWoS packaging technology. However, with the surge in demand for generative AI, TSMC’s CoWoS capacity is severely strained. Consequently, NVIDIA has certified other CoWoS packaging suppliers such as UMC, ASE, and American OSAT manufacturer Amkor as backup options.

Meanwhile, TSMC has relocated its InFo production capacity from Longtan to Southern Taiwan Science Park. The vacated Longtan fab is being repurposed to expand CoWoS capacity, while the Zhunan and Taichung fabs are also contributing to the expansion of CoWoS production to alleviate capacity constraints.

However, during the earnings call, TSMC also stated that despite a doubling of capacity in 2024, it still may not be sufficient to meet all customer demands.

In addition to TSMC’s CoWoS capacity, industry rumors suggest that NVIDIA has made significant upfront payments to Micron, SK Hynix, to secure HBM3 memory, ensuring a stable supply of HBM memory. However, the entire HBM capacity of Samsung, SK Hynix, and Micron for this year has already been allocated. Therefore, whether the capacity can meet market demand will be a significant challenge for NVIDIA.

• Internal Concern 2: Major Customers Shifting Towards In-house Chips

While cloud service providers (CSPs) fiercely compete for GPUs, major players like Amazon, Microsoft, Google, and Meta are actively investing in in-house AI chips.

Amazon and Google have respectively introduced Trainium and TPU chips, Microsoft announced its first in-house AI chip Maia 100 along with in-house cloud computing CPU Cobalt 100, while Meta plans to unveil its first-generation in-house AI chip MTIA by 2025.

Although these hyperscale customers still rely on NVIDIA’s chips, in the long run, it may impact NVIDIA’s market share, inadvertently positioning them as competitors and affecting profits. Consequently, NVIDIA finds it challenging to depend solely on these hyperscale customers.

• External Challenge 1: Export Control Pressures Lead to Loss of Chinese Customers

Due to escalating tensions between the US and China, the US issued new regulations prohibiting NVIDIA from exporting advanced AI chips to China. Consequently, NVIDIA introduced specially tailored versions such as A800 and H800 for the Chinese market.

However, they were ultimately blocked by the US, and products including A100, A800, H100, H800, and L40S were included in the export control list.Subsequently, NVIDIA decided to introduce new AI GPUs, namely HGXH20, L20 PCIe, and L2 PCIe, in compliance with export policies.

However, with only 20% of the computing power of H100, they are planned for mass production in the second quarter. Due to the reduced performance, major Chinese companies like Alibaba, Tencent, and Baidu reportedly refused to purchase, explicitly stating significant order cuts for the year. Consequently, NVIDIA’s revenue prospects in China appear grim, with some orders even being snatched by Huawei.

Currently, NVIDIA’s sales revenue from Singapore and China accounts for 15% of its total revenue. Moreover, the company holds over 90% market share in the AI chip market in China. Therefore, the cost of abandoning the Chinese market would be substantial. NVIDIA is adamant about not easily giving up on China; however, the challenge lies in how to comply with US government policies and pressures while meeting the demands of Chinese customers.

As per NVIDIA CEO Jensen Huang during its last earnings call, he mentioned that US export control measures would have an impact. Contributions from China and other regions accounted for 20-25% of data center revenue in the last quarter, with a significant anticipated decline this quarter.

He also expressed concerns that besides losing the Chinese market, the situation would accelerate China’s efforts to manufacture its own chips and introduce proprietary GPU products, providing Chinese companies with opportunities to rise.

• External Challenge 2: Arch-Rivals Intel and AMD Begin Their Offensive

In the race to capture the AI market opportunity, arch-rivals Intel and AMD are closely after NVIDIA. As NVIDIA pioneered the adoption of TSMC’s 4-nanometer H100, AMD quickly followed suit by launching the first batch of “Instinct MI300X” for AI and HPC applications last year.

Currently, shipments of MI300X have commenced this year, with Microsoft’s data center division emerging as the largest buyer. Meta has also procured a substantial amount of Instinct MI300 series products, while LaminiAI stands as the first publicly known company to utilize MI300X.

According to official performance tests by AMD, the MI300X outperforms the existing NVIDIA H100 80GB available on the market, posing a potential threat to the upcoming H200 141GB.

Additionally, compared to the H100 chip, the MI300X offers a more competitive price for products of the same level. If NVIDIA’s production capacity continues to be restricted, some customers may switch to AMD.

Meanwhile, Intel unveiled the “Gaudi3” chip for generative AI software last year. Although there is limited information available, it is rumored that the memory capacity may increase by 50% compared to Gaudi 2’s 96GB, possibly upgrading to HBM3e memory. CEO Pat Gelsinger directly stated that “Gaudi 3 performance will surpass that of the H100.”

• External Challenge 3: Startup Underdogs Form AI Platform Alliance in Attempt to Conquer

Several global chip design companies have recently announced the formation of the “AI Platform Alliance,” aiming to promote an open AI ecosystem. The founding members of the AI Platform Alliance include Ampere, Cerebras Systems, Furiosa, Graphcore, Kalray, Kinara, Luminous, Neuchips, Rebellions, and Sapeon, among others.

Notably absent is industry giant NVIDIA, leading to speculation that startups aspire to unite and challenge NVIDIA’s dominance.

However, with NVIDIA holding a 75-90% market share in AI, it remains in a dominant position. Whether the AI Platform Alliance can disrupt NVIDIA’s leading position is still subject to observation.

Read more

• [News] NVIDIA’s Exclusive Chips for China Now Reported to be Available for Pre-Order, Priced Similar to Huawei Products
• [News] NVIDIA CEO Jensen Huang Meets with TSMC Executives in Taiwan, Addressing the Significant AI Chip Shortage

(Photo credit: NVIDIA)