Over the past few decades, semiconductor manufacturing technology has evolved from the 10,000nm process in 1971 to the 3nm process in 2022, driven by the need to increase the number of transistors on chips for enhanced computational performance. However, as applications like artificial intelligence (AI) and AIGC rapidly advance, demand for higher core chip performance at the device level is growing.

While process technology improvements may encounter bottlenecks, the need for computing resources continues to rise. This underscores the importance of advanced packaging techniques to boost the number of transistors on chips.

In recent years, “advanced packaging” has gained significant attention. Think of “packaging” as a protective shell for electronic chips, safeguarding them from adverse environmental effects. Chip packaging involves fixation, enhanced heat dissipation, electrical connections, and signal interconnections with the outside world. The term “advanced packaging” primarily focuses on packaging techniques for chips with process nodes below 7nm.

Amid the AI boom, which has driven demand for AI servers and NVIDIA GPU graphics chips, CoWoS (Chip-on-Wafer-on-Substrate) packaging has faced a supply shortage.

But what exactly is CoWoS?

CoWoS is a 2.5D and 3D packaging technology, composed of “CoW” (Chip-on-Wafer) and “WoS” (Wafer-on-Substrate). CoWoS involves stacking chips and then packaging them onto a substrate, creating a 2.5D or 3D configuration. This approach reduces chip space, while also lowering power consumption and costs. The concept is illustrated in the diagram below, where logic chips and High-Bandwidth Memory (HBM) are interconnected on an interposer through tiny metal wires. “Through-Silicon Vias (TSV)” technology links the assembly to the substrate beneath, ultimately connecting to external circuits via solder balls.

The difference between 2.5D and 3D packaging lies in their stacking methods. 2.5D packaging involves horizontal chip stacking on an interposer or through silicon bridges, mainly for combining logic and high-bandwidth memory chips. 3D packaging vertically stacks chips, primarily targeting high-performance logic chips and System-on-Chip (SoC) designs.

When discussing advanced packaging, it’s worth noting that Taiwan Semiconductor Manufacturing Company (TSMC), rather than traditional packaging and testing facilities, is at the forefront. CoW, being a precise part of CoWoS, is predominantly produced by TSMC. This situation has paved the way for TSMC’s comprehensive service offerings, which maintain high yields in both fabrication and packaging processes. Such a setup ensures an unparalleled approach to serving high-end clients in the future.

Applications of CoWoS

The shift towards multiple small chips and memory stacking is becoming an inevitable trend for high-end chips. CoWoS packaging finds application in a wide range of fields, including High-Performance Computing (HPC), AI, data centers, 5G, Internet of Things (IoT), automotive electronics, and more. In various major trends, CoWoS packaging is set to play a vital role.

In the past, chip performance was primarily reliant on semiconductor process improvements. However, with devices approaching physical limits and chip miniaturization becoming increasingly challenging, maintaining small form factors and high chip performance has required improvements not only in advanced processes but also in chip architecture. This has led to a transition from single-layer chips to multi-layer stacking. As a result, advanced packaging has become a key driver in extending Moore’s Law and is leading the charge in the semiconductor industry.

(Photo credit: TSMC)

South Korean media reported that Samsung is set to manufacture a new generation of Full Self-Driving (FSD) chips for Tesla’s Level 5 autonomous vehicles. These chips will be utilized in Tesla Hardware 5 (HW 5.0) onboard computers, with production expected to commence after 2025. The chips will be manufactured using Samsung’s 4nm process.

TrendForce’s analysis:

Samsung May Competing with TSMC for Tesla HW 5.0 Chips

In the early stages of Tesla’s autonomous driving technology, the company collaborated with Samsung for FSD chips used in various vehicle models, including Model 3, Model 5, Model X, and Model Y. However, in 2022, Tesla chose to work with TSMC, citing TSMC’s better yield performance in 4nm process technology at that time.

In response, Samsung has been actively improving its 3nm and 4nm process technologies within a short period. While Samsung’s 4nm process yield has reached 75%, it still slightly lags behind TSMC’s 80%. Despite this difference, given their previous collaborations, it is not ruled out that Tesla might place orders with both TSMC and Samsung this time. The main reason being Samsung’s plan to advance to the 2nm-level SF2 process technology in 2025 and further progress to the 1.4nm-level SF1.4 process technology in 2027, aligning its overall roadmap with TSMC’s. This advancement will assist Tesla in accelerating the production plan of its DOJO supercomputer, facilitating the transition to Level 5 autonomous driving.

(Photo credit: Tesla)

Looking at the impact of AI server development on the PCB industry, mainstream AI servers, compared to general servers, incorporate 4 to 8 GPUs. Due to the need for high-frequency and high-speed data transmission, the number of PCB layers increases, and there’s an upgrade in the adoption of CCL grade as well. This surge in GPU integration drives the AI server PCB output value to surpass that of general servers by several times. However, this advancement also brings about higher technological barriers, presenting an opportunity for high-tech PCB manufacturers to benefit.

TrendForce’s perspective: 

• The increased value of AI server PCBs primarily comes from GPU boards.

Taking the NVIDIA DGX A100 as an example, its PCB can be divided into CPU boards, GPU boards, and accessory boards. The overall value of the PCB is about 5 to 6 times higher than that of a general server, with approximately 94% of the incremental value attributed to the GPU boards. This is mainly due to the fact that general servers typically do not include GPUs, while the NVIDIA DGX A100 is equipped with 8 GPUs.

Further analysis reveals that CPU boards, which consist of CPU boards, CPU mainboards, and functional accessory boards, make up about 20% of the overall AI server PCB value. On the other hand, GPU boards, including GPU boards, NV Switch, OAM (OCP Accelerator Module), and UBB (Unit Baseboard), account for around 79% of the total AI server PCB value. Accessory boards, composed of components such as power supplies, HDD, and cooling systems, contribute to only about 1% of the overall AI server PCB value.

• The technological barriers of AI servers are rising, leading to a decrease in the number of suppliers.

Since AI servers require multiple card interconnections with more extensive and denser wiring compared to general servers, and AI GPUs have more pins and an increased number of memory chips, GPU board assemblies may reach 20 layers or more. With the increase in the number of layers, the yield rate decreases.

Additionally, due to the demand for high-frequency and high-speed transmission, CCL materials have evolved from Low Loss grade to Ultra Low Loss grade. As the technological barriers rise, the number of manufacturers capable of entering the AI server supply chain also decreases.

Currently, the suppliers for CPU boards in AI servers include Ibiden, AT&S, Shinko, and Unimicron, while the mainboard PCB suppliers consist of GCE and Tripod. For GPU boards, Ibiden serves as the supplier, and for OAM PCBs, Unimicron and Zhending are the suppliers, with GCE, ACCL, and Tripod currently undergoing certification. The CCL suppliers include EMC. For UBB PCBs, the suppliers are GCE, WUS, and ACCL, with TUC and Panasonic being the CCL suppliers.

Regarding ABF boards, Taiwanese manufacturers have not yet obtained orders for NVIDIA AI GPUs. The main reason for this is the limited production volume of NVIDIA AI GPUs, with an estimated output of only about 1.5 million units in 2023. Additionally, Ibiden’s yield rate for ABF boards with 16 layers or more is approximately 10% to 20% higher than that of Taiwanese manufacturers. However, with TSMC’s continuous expansion of CoWoS capacity, it is expected that the production volume of NVIDIA AI GPUs will reach over 2.7 million units in 2024, and Taiwanese ABF board manufacturers are likely to gain a low single-digit percentage market share.

(Photo credit: Google)

TSMC held its Q2 earnings conference today (20th). TrendForce utilized AI tools to transcribe and made slight edits, presenting the full text of TSMC’s operational status report delivered by Chairman Mark Liu, CEO C.C. Wei, and CFO Wendell Huang. …

According to media reports, in response to the booming demand in the artificial intelligence market, TSMC has altered its Kaohsiung factory plan. Originally scheduled for a 28-nanometer mature process, the factory will now be equipped with a 2-nanometer advanced process, with mass production expected to commence in the latter half of 2025. The official announcement of this factory plan is imminent.

During a investor conference held on July 20th, TSMC refrained from making any comments, citing the current quiet period. As reported by “Central News Agency,” Kaohsiung Mayor Chen Chi-mai expressed the city government’s respect for TSMC and pledged full assistance. However, it is worth noting that the 2-nanometer process requires more funding compared to the 28-nanometer process, and TSMC has already informed the Kaohsiung city government, seeking support in terms of water and power supply.

Official data indicates that TSMC’s 2-nanometer process offers a 10% to 15% performance improvement at the same power consumption or a 20% to 30% reduction in power consumption at the same performance level compared to the 3-nanometer process. The primary production base for the 2-nanometer process will be located in Hsinchu’s Baoshan area, with plans to construct four fabs. The trial production is scheduled for 2024, followed by mass production in the latter half of 2025.

(Photo credit: TSMC)