The semiconductor industry enters the era of integration. Various foundries are focusing on advanced packaging technologies, but the terminology surrounding advanced packaging can be daunting. This article aims to explain these terms in the simplest way possible.

According to a report from TechNews, currently, there are two main trends in advanced packaging: heterogeneous integration and chiplets.

In fact, the concept of “heterogeneous integration” has been developing for many years and is not exclusive to advanced packaging. It is not only used for the integration of heterogeneous chiplets but also for integrating other non-chip active/passive components into a single package, which is the technology commonly used in traditional Outsourced Semiconductor Assembly and Test Services(OSATs).

• Heterogeneous Integration = Big Building Blocks? Advanced Packaging = Small Building Blocks? 

In the simplest terms, “heterogeneous integration” can be likened to building with large building blocks, while “advanced packaging” is akin to assembling with small building blocks. Some manufacturers, like traditional Outsourced Semiconductor Assembly and Test Services(OSATs), excel in stacking large blocks, such as logic circuits, radio frequency circuits, MEMS (Micro-Electro-Mechanical Systems), or sensors, onto a IC substrate. The stacking of these different large blocks represents the concept of heterogeneous integration.

On the other hand, some blocks are too small to stack effectively, requiring assistance from advanced packaging, typically provided by semiconductor foundries.

Advanced packaging also encompasses 2.5D packaging and 3D packaging. Using the metaphor of building blocks, the former involves horizontally stacking small building blocks on a interposer, while the latter involves vertically stacking small building blocks with interconnection facilitated through Through-Silicon Vias (TSVs), which are ultra-small building blocks.

It’s important to emphasize that stacking blocks is a conceptual representation, and the distinction between large and small blocks is relative. The analogy above refers to heterogeneous integration in traditional packaging, and heterogeneous integration in advanced packaging follows a similar concept, but with even smaller building blocks.

• SoC / SiP / Heterogeneous Integration / Chiplet

With this concept in mind, let’s discuss the applications of heterogeneous integration in advanced packaging:

Among the various packaging types, SoC (System On Chip) involves integrating different chips such as processors and memory, with different functions, redesigned and fabricated using the “same process,” integrated onto a single chip, resulting in a final product with only one chip.

On the other hand, SiP (System in Package) involves connecting multiple chips with “different processes” through “heterogeneous integration” technology, integrated within the same packaging module. Therefore, the final product will be a system with many chips on it, resembling the stacking of different-sized building blocks mentioned earlier.

Therefore, heterogeneous integration refers to integrating different and separately manufactured components (heterogeneous) into higher-level assemblies. These components include blocks of different sizes, such as MEMS devices, passive components, logic chips, and more.

However, at a certain point, for the sake of process development, researchers found that separating components at the right time might facilitate miniaturization. Hence, chiplet was born.

• Is Chiplet the Fusion of Heterogeneous Integration and Advanced Processes?

As demands for ICs become increasingly complex, the size of SoC chips continues to grow. However, cramming too many components onto a limited substrate poses significant challenges, including heightened process complexity and reduced yield.

Hence, the concept of chiplets emerged, advocating for the segmentation of SoC functionalities, such as data storage, computation, signal processing, and data flow management, into smaller individual chips. These chiplets are then integrated through packaging to form a interconnected network.

It’s worth noting that Chiplets are essentially chips, whereas SiP refers to the packaging format. Chiplet architecture enable the reduction of individual chip sizes, simplify circuit design, overcome manufacturing difficulties and yield issues, and offer greater design flexibility.

• Advanced Packaging Technologies in Foundries

Currently, advanced packaging can be broadly categorized into three main types: Wafer-Level Packaging (WLP), 2.5D Packaging, and 3D Packaging. Traditional packaging involves cutting wafers into chips before packaging, while advanced packaging entails packaging the silicon wafer before cutting, requiring subsequent stacking processes in fabs. Therefore, the technology is primarily the responsibility of fabs.

Traditional packaging involves cutting wafers into chips before packaging. Advanced packaging, starting from wafer-level packaging, involves packaging silicon wafers before cutting, and subsequent stacking requires wafer fabrication processes.

Therefore, this article will delve into advanced packaging technologies offered by the three major foundries, with a focus on 2.5D and 3D packaging.

• 2.5DIC / 3DIC Packaging

To further explain using building blocks, the difference between 2.5D and 3DIC packaging lies in the “stacking method.”

In 2.5D packaging, processors, memory, or other chips are stacked horizontally on a silicon interposer using a flip-chip method, with micro bumps connecting different chip’s electronic signals. Through silicon vias (TSVs) in the interposer link to the metal bumps below, then packaged onto the IC substrate, creating tighter interconnections between the chips and the substrate.

In a side view, although the chips are stacked, the essence remains horizontal packaging, with the chips positioned closer together and allowing for smaller chip sizes. Additionally, this is a form of “heterogeneous integration” technology.

3D packaging involves stacking multiple chips (face down) together, directly using through-silicon vias to stack them vertically, linking the electronic signals of different chips above and below, achieving true vertical packaging. Currently, more and more CPUs, GPUs, and memories are starting to adopt 3D packaging technology.

• Hybrid Bonding

Hybrid bonding is one of the die bonding techniques used in advanced chip packaging processes. One of the commercially available technologies in this domain is the “Cu-Cu hybrid bonding.”

In traditional wafer bonding processes, there are interfaces between copper and dielectric materials. With “Cu-Cu hybrid bonding,” metal contacts are embedded within the dielectric material. Through a thermal treatment process, these two materials are bonded together, utilizing the atomic diffusion of copper metal in its solid-state to achieve the bond. This approach addresses challenges encountered in previous flip-chip bonding process.

Compared to flip-chip bonding, hybrid bonding offers several advantages. It allows for achieving ultra-high I/O counts and longer interconnect lengths. By using dielectric material for bonding instead of bottom fillers, the cost of filling is eliminated.

Additionally, hybrid bonding results in minimal thickness compared to chip-on-wafer bonding. This is particularly beneficial for future developments in 3D packaging, where stacking multiple layers of chips is required, as hybrid bonding can significantly reduce the overall thickness.

• Advanced Packaging Moves Towards the Era of Heterogeneous Integration

As the semiconductor industry enters the “post-Moore’s Law era,” the development focus of advanced packaging is gradually shifting from 2D planar structures to 3D stacking and from single-chip designs to multi-chip configurations. Therefore, “heterogeneous integration” will play a crucial role in future advanced packaging.

Currently, prominent companies such as TSMC, Samsung, and Intel are intensifying their research and development efforts and capacity expansions in this field, introducing their innovative packaging solutions.

With ongoing technological advancements and innovations, advanced packaging and heterogeneous integration will play increasingly vital roles in propelling the semiconductor industry towards greater heights, meeting the complex and diverse demands of future electronic devices.

Read more

• [News] The Era of Heterogeneous Integration Approaches: Who Shall Dominate the Advanced Packaging Field?

(Photo credit: Intel)

Samsung, the Korean tech giant, has unveiled SAINT technology to counter TSMC’s advanced CoWoS packaging, aiming to benefit from the surging AI market. Market reports reveal that Samsung is strategically procuring a substantial amount of 2.5D packaging equipment, indicating a keen awareness of the soaring demand from AI chip companies like NVIDIA, reported by Korean media The Elec.

Samsung has acquired 16 sets of packaging equipment from the Japanese company Shinkawa. Currently, 7 sets have been received, with the possibility of additional orders based on future requirements. Samsung’s objective is to demonstrate its prowess in packaging and HBM technologies, seeking recognition and partnership with NVIDIA. As the limitations in NVIDIA’s current supply chain, especially due to insufficient CoWoS advanced packaging capacity in TSMC, Samsung emerges as a promising alternative for diverse supply chain.

On the other hand, NVIDIA’s ambitious goal of achieving USD 300 billion in AI sector revenue by 2027 requires a reliable supply chain, as per reported by TechNews. To this end, Samsung is poised to supply its next-gen GPU, Blackwell, featuring HBM3 and 2.5D packaging. This move aligns with NVIDIA’s strategy to diversify its supply chain away from existing providers like TSMC.

For Samsung, this collaboration presents a significant opportunity to enter the thriving AI market. Success in this venture could not only bolster the financial performance of Samsung’s memory and advanced packaging divisions but also open doors to orders from players like AMD and Tesla. However, the key lies in how effectively Samsung meets the formidable market demand, particularly in semiconductor production, advanced packaging, and memory technology.

(Image: Samsung)

Explore more

• [News] TSMC’s CoWoS Demand Surges from NVIDIA, Apple, AMD, Broadcom, Marvell, Monthly Capacity Up 120% in 2024

As applications like AIGC, 8K, AR/MR, and others continue to develop, 3D IC stacking and heterogeneous integration of chiplet have become the primary solutions to meet future high-performance computing demands and extend Moore’s Law.

Major companies like TSMC and Intel have been expanding their investments in heterogeneous integration manufacturing and related research and development in recent years. Additionally, leading EDA company Cadence has taken the industry lead by introducing the “Integrity 3D-IC” platform, an integrated solution for design planning, realization, and system analysis simulation tools, marking a significant step towards 3D chip stacking.

Differences between 2.5D and 3D Packaging

The main difference between 2.5D and 3D packaging technologies lies in the stacking method. 2.5D packaging involves stacking chips one by one on an interposer or connecting them through silicon bridges, primarily used for assembling logic processing chips and high-bandwidth memory. On the other hand, 3D packaging is a technology that vertically stacks chips, mainly targeting high-performance logic chips and SoC manufacturing.

CPU and HBM Stacking Demands

With the rapid development of applications like AIGC, AR/VR, and 8K, it is expected that a significant amount of computational demand will arise, particularly driving the need for parallel computing systems capable of processing big data in a short time. To overcome the bandwidth limitations of DDR SDRAM and further enhance parallel computing performance, the industry has been increasingly adopting High-Bandwidth Memory (HBM). This trend has led to a shift from the traditional “CPU + memory (such as DDR4)” architecture to the “Chip + HBM stacking” 2.5D architecture. With continuous growth in computational demand, the future may see the integration of CPU, GPU, or SoC through 3D stacking.

3D Stacking with HBM Prevails, but CPU Stacking Lags Behind

HBM was introduced in 2013 as a 3D stacked architecture for high-performance SDRAM. Over time, the stacking of multiple layers of HBM has become widespread in packaging, while the stacking of CPUs/GPUs has not seen significant progress.

The main reasons for this disparity can be attributed to three factors: 1. Thermal conduction, 2. Thermal stress, and 3. IC design. First, 3D stacking has historically performed poorly in terms of thermal conduction, which is why it is primarily used in memory stacking, as memory operations generate much less heat than logic operations. As a result, the thermal conduction issues faced by current memory stacking products can be largely disregarded.

Second, thermal stress issues arise from the mismatch in coefficients of thermal expansion (CTE) between materials and the introduction of stress from thinning the chips and introducing metal layers. The complex stress distribution in stacked structures has a significant negative impact on product reliability.

Finally, IC design challenges from a lack of EDA tools, as traditional CAD tools are inadequate for handling 3D design rules. Developers must create their own tools to address process requirements, and the complex design of 3D packaging further increases the design, manufacturing, and testing costs.

How EDA Companies Offer Solutions

Cadence, during the LIVE Taiwan 2023 user annual conference, highlighted its years of effort in developing solutions. They have introduced tools like the Clarity 3D solver, Celsius thermal solver, and Sigrity Signal and Power Integrity, which can address thermal conduction and thermal stress simulation issues. When combined with Cadence’s comprehensive EDA tools, these offerings contribute to the growth of the “Integrity 3D-IC” platform, aiding in the development of 3D IC design.

“3D IC” represents a critical design trend in semiconductor development. However, it presents greater challenges and complexity than other projects. In addition to the challenges in Logic IC design, there is a need for analog and multi-physics simulations. Therefore, cross-platform design tools are indispensable. The tools provided by EDA leader Cadence are expected to strengthen the 3D IC design tool platform, reducing the technological barriers for stacking CPU, GPU, or SoC to enhance chip computing performance.

This article is from TechNews, a collaborative media partner of TrendForce.

(Photo credit: TSMC)

With the advancements in AIGC models such as ChatGPT and Midjourney, we are witnessing the rise of more super-sized language models, opening up new possibilities for High-Performance Computing (HPC) platforms.

According to TrendForce, by 2025, the global demand for computational resources in the AIGC industry – assuming 5 super-sized AIGC products equivalent to ChatGPT, 25 medium-sized AIGC products equivalent to Midjourney, and 80 small-sized AIGC products – would be approximately equivalent to 145,600 – 233,700 units of NVIDIA A100 GPUs. This highlights the significant impact of AIGC on computational requirements.

Additionally, the rapid development of supercomputing, 8K video streaming, and AR/VR will also lead to an increased workload on cloud computing systems. This calls for highly efficient computing platforms that can handle parallel processing of vast amounts of data.
However, a critical concern is whether hardware advancements can keep pace with the demands of these emerging applications.

HBM: The Fast Lane to High-Performance Computing

While the performance of core computing components like CPUs, GPUs, and ASICs has improved due to semiconductor advancements, their overall efficiency can be hindered by the limited bandwidth of DDR SDRAM.

For example, from 2014 to 2020, CPU performance increased over threefold, while DDR SDRAM bandwidth only doubled. Additionally, the pursuit of higher transmission performance through technologies like DDR5 or future DDR6 increases power consumption, posing long-term impacts on computing systems’ efficiency.

Recognizing this challenge, major chip manufacturers quickly turned their attention to new solutions. In 2013, AMD and SK Hynix made separate debuts with their pioneering products featuring High Bandwidth Memory (HBM), a revolutionary technology that allows for stacking on GPUs and effectively replacing GDDR SDRAM. It was recognized as an industry standard by JEDEC the same year.

In 2015, AMD introduced Fiji, the first high-end consumer GPU with integrated HBM, followed by NVIDIA’s release of P100, the first AI server GPU with HBM in 2016, marking the beginning of a new era for server GPU’s integration with HBM.

HBM’s rise as the mainstream technology sought after by key players can be attributed to its exceptional bandwidth and lower power consumption when compared to DDR SDRAM. For example, HBM3 delivers 15 times the bandwidth of DDR5 and can further increase the total bandwidth by adding more stacked dies. Additionally, at system level, HBM can effectively manage power consumption by replacing a portion of GDDR SDRAM or DDR SDRAM.

As computing power demands increase, HBM’s exceptional transmission efficiency unlocks the full potential of core computing components. Integrating HBM into server GPUs has become a prominent trend, propelling the global HBM market to grow at a compound annual rate of 40-45% from 2023 to 2025, according to TrendForce.

The Crucial Role of 2.5D Packaging

In the midst of this trend, the crucial role of 2.5D packaging technology in enabling such integration cannot be overlooked.

TSMC has been laying the groundwork for 2.5D packaging technology with CoWoS (Chip on Wafer on Substrate) since 2011. This technology enables the integration of logic chips on the same silicon interposer. The third-generation CoWoS technology, introduced in 2016, allowed the integration of logic chips with HBM and was adopted by NVIDIA for its P100 GPU.

With development in CoWoS technology, the interposer area has expanded, accommodating more stacked HBM dies. The 5th-generation CoWoS, launched in 2021, can integrate 8 HBM stacks and 2 core computing components. The upcoming 6th-generation CoWoS, expected in 2023, will support up to 12 HBM stacks, meeting the requirements of HBM3.

TSMC’s CoWoS platform has become the foundation for high-performance computing platforms. While other semiconductor leaders like Samsung, Intel, and ASE are also venturing into 2.5D packaging technology with HBM integration, we think TSMC is poised to be the biggest winner in this emerging field, considering its technological expertise, production capacity, and order capabilities.

In conclusion, the remarkable transmission efficiency of HBM, facilitated by the advancements in 2.5D packaging technologies, creates an exciting prospect for the seamless convergence of these innovations. The future holds immense potential for enhanced computing experiences.

The surge in AIGC and new technologies such IoT, AI, 5G, AR/VR are driving a huge demand for computational power of high-end chips. This has been even outpacing the performance increase offered by the long-standing Moore’s Law, ushering in a “post-Moore” era where revolutions in advanced chip design are crucial.

Over recent years, chiplet design has seemingly become the mainstream approach for upgrading high-end chips. The concept is to allow more transistors on a single chip, effectively increasing the production yield of high-end chips while reducing overall costs.

By the large, major IC players have all jumped on board. Even Apple has joined the game by releasing their M1 Ultra SoC using the chiplet concept, which doubles computational performance by integrating two M1 Max units in a single chip.

The CPU sector is definitely a clear demonstration of this trend:

• AMD took the leap with chiplet design in their 2nd-gen EPYC CPUs, doubling the computing cores from 32 to 64 within two years, while slashing costs by up to half. The company has extended this approach to their 4th-gen EPYC CPUs and even pioneered the GPU Navi 31, the first of its kind to use chiplets.
• Intel started incorporating chiplets into their Lakefield series SoC in 2020. Looking ahead, their upcoming CPUs like the Meteor Lake set for 2023, and Arrow Lake and Lunar Lake scheduled for 2024, will all use chiplet design.

Transition from Bumping to Hybrid Bonding

Our analysis in “Chiplet Design: A Real Game-Changer for Substrates” laid out the comprehensive impact of the evolution of chiplet technology on substrates. In fact, chiplets have already caused a significant disruption to the most advanced semiconductor packaging technologies, necessitating the transition towards advanced 2.5D and 3D packaging technologies.

The bottleneck of advanced packaging lies in the chiplets’ interconnections, with bump and microbump still being the key technology for linking chips and forming I/O joints. These connection densities are hard to enhance, thus limiting the overall chip’s transmission speed. In addition, the more chiplets being stacked, the bigger the chip volume gets. The challenge is how to limit the chip size within a specific range, considering the current technical constraints.

Therefore, copper-to-copper hybrid bonding, also known as DBI (Direct Bond Interconnect), has been emerging as the key technology route that overcomes major hurdles in chiplet integration from the bottom-up.

Unlike bumping technology, hybrid bonding significantly shrinks the I/O joint space. The future transmission demand requires the I/O joint space between chiplets to be less than 10µm. While bumping is limited to around 20µm, hybrid bonding can take this down to an impressive 1µm or even less. This also means more I/O joints can be fitted in the same chip size – even reaching up to millions on a mere 1cm2 chip.

On top of this, hybrid bonding only adds an extra 1-2µm of thickness, compared to the 10-30µm of microbump, thereby helping reduce the thickness of stacked chips.

To put it simply, hybrid bonding can boost transmission efficiency, minimize energy usage with higher density of copper joints, manage chip volume, and even cut down on material costs.

The Race for Advanced Packaging Is Kicking Off

Moving forward, hybrid bonding is set to become the key technology supporting the continuous development of chiplet design and 3D packaging. This has been exemplified by TSMC’s front-end So IC packaging technology which is based on hybrid bonding. This puts AMD, a key customer of TSMC, in a favorable position to get ahead.

From AMD’s roadmap of 3D V-Cache technology, they have stacked SRAM on top of CCX (CPU Complex), and gradually integrated it into Milan-X series, the EPYC server CPUs, and Ryzen series, the consumer-grade CPUs, over the past two years. This has significantly improved performance and power consumption as a whole.

Not to be outdone, this year Intel also launched their Foveros Direct packaging technology, which is also based on hybrid bonding route. Assuming everything proceeds smoothly, we can anticipate the release of CPUs utilizing Foveros Direct technology by 2024.

As we look at the current products, AMD’s hybrid bonding apparently focuses on stacking SRAM and computing units at the moment. However, as CPU leaders deepen their understanding of this technology, the application field is expected to further expand. In other words, the future of hybrid bonding solutions stacking multiple computation units is just around the corner.