According to TrendForce’s latest report on the server industry, not only have emerging applications in recent years accelerated the pace of AI and HPC development, but the complexity of models built from machine learning applications and inferences that involve increasingly sophisticated calculations has also undergone a corresponding growth as well, resulting in more data to be processed. While users are confronted with an ever-growing volume of data along with constraints placed by existing hardware, they must make tradeoffs among performance, memory capacity, latency, and cost. HBM (High Bandwidth Memory) and CXL (Compute Express Link) have thus emerged in response to the aforementioned conundrum. In terms of functionality, HBM is a new type of DRAM that addresses more diverse and complex computational needs via its high I/O speeds, whereas CXL is an interconnect standard that allows different processors, or xPUs, to more easily share the same memory resources.

HBM breaks through bandwidth limitations of traditional DRAM solutions through vertical stacking of DRAM dies

Memory suppliers developed HBM in order to be free from the previous bandwidth constraints posed by traditional memory solutions. Regarding memory architecture, HBM consists of a base logic die with DRAM dies vertically stacked on top of the logic die. The 3D-stacked DRAM dies are interconnected with TSV and microbumps, thereby enabling HBM’s high-bandwidth design. The mainstream HBM memory stacks involve four or eight DRAM die layers, which are referred to as “4-hi” or “8-hi”, respectively. Notably, the latest HBM product currently in mass production is HBM2e. This generation of HBM contains four or eight layers of 16Gb DRAM dies, resulting in a memory capacity of 8GB or 16GB per single HBM stack, respectively, with a bandwidth of 410-460GB/s. Samples of the next generation of HBM products, named HBM3, have already been submitted to relevant organizations for validation, and these products will likely enter mass production in 2022.

TrendForce’s investigations indicate that HBM comprises less than 1% of total DRAM bit demand for 2021 primarily because of two reasons. First, the vast majority of consumer applications have yet to adopt HBM due to cost considerations. Second, the server industry allocates less than 1% of its hardware to AI applications; more specifically, servers that are equipped with AI accelerators account for less than 1% of all servers currently in use, not to mention the fact that most AI accelerators still use GDDR5(x) and GDDR6 memories, as opposed to HBM, to support their data processing needs.

Although HBM currently remains in the developmental phase, as applications become increasingly reliant on AI usage (more precise AI needs to be supported by more complex models), computing hardware will then require the integration of HBM to operate these applications effectively. In particular, FPGA and ASIC represent the two hardware categories that are most closely related to AI development, with Intel’s Stratix and Agilex-M as well as Xilinx’s Versal HBM being examples of FPGA with onboard HBM. Regarding ASIC, on the other hand, most CSPs are gradually adopting their own self-designed ASICs, such Google’s TPU, Tencent’s Enflame DTU, and Baidu’s Kunlun – all of which are equipped with HBM – for AI deployments. In addition, Intel will also release a high-end version of its Sapphire Rapids server CPU equipped with HBM by the end of 2022. Taking these developments into account, TrendForce believes that an increasing number of HBM applications will emerge going forward due to HBM’s critical role in overcoming hardware-related bottlenecks in AI development.

A new memory standard born out of demand from high-speed computing, CXL will be more effective in integrating resources of whole system

Evolved from PCIe Gen5, CXL is a memory standard that provides high-speed and low-latency interconnections between the CPU and other accelerators such as the GPU and FPGA. It enables memory virtualization so that different devices can share the same memory pool, thereby raising the performance of a whole computer system while reducing its cost. Hence, CXL can effectively deal with the heavy workloads related to AI and HPC applications.

CXL is just one of several interconnection technologies that feature memory sharing. Other examples that are also in the market include NVLink from NVIDIA and Gen-Z from AMD and Xilinx. Their existence is an indication that the major ICT vendors are increasingly attentive to the integration of various resources within a computer system. TrendForce currently believes that CXL will come out on top in the competition mainly because it is introduced and promoted by Intel, which has an enormous advantage with respect to the market share for CPUs. With Intel’s support in the area of processors, CXL advocates and hardware providers that back the standard will be effective in organizing themselves into a supply chain for the related solutions. The major ICT companies that have in turn joined the CXL Consortium include AMD, ARM, NVIDIA, Google, Microsoft, Facebook (Meta), Alibaba, and Dell. All in all, CXL appears to be the most favored among memory protocols.

The consolidation of memory resources among the CPU and other devices can reduce communication latency and boost the computing performance needed for AI and HPC applications. For this reason, Intel will provide CXL support for its next-generation server CPU Sapphire Rapids. Likewise, memory suppliers have also incorporated CXL support into their respective product roadmaps. Samsung has announced that it will be launching CXL-supported DDR5 DRAM modules that will further expand server memory capacity so as to meet the enormous resource demand of AI computing. There is also a chance that CXL support will be extended to NAND Flash solutions in the future, thus benefiting the development of both types of memory products.

Synergy between HBM and CXL will contribute significantly to AI development; their visibility will increase across different applications starting in 2023

TrendForce believes that the market penetration rate of CXL will rise going forward as this interface standard is built into more and more CPUs. Also, the combination of HBM and CXL will be increasingly visible in the future hardware designs of AI servers. In the case of HBM, it will contribute to a further ramp-up of data processing speed by increasing the memory bandwidth of the CPU or the accelerator. As for CXL, it will enable high-speed interconnections among CPU and other devices. By working together, HBM and CXL will raise computing power and thereby expedite the development of AI applications.

The latest advances in memory pooling and sharing will help overcome the current hardware bottlenecks in the designs of different AI models and continue the trend of more sophisticated architectures. TrendForce anticipates that the adoption rate of CXL-supported Sapphire Rapids processors will reach a certain level, and memory suppliers will also have put their HBM3 products and their CXL-supported DRAM and SSD products into mass production. Hence, examples of HBM-CXL synergy in different applications will become increasingly visible from 2023 onward.

For more information on reports and market data from TrendForce’s Department of Semiconductor Research, please click here, or email Ms. Latte Chung from the Sales Department at lattechung@trendforce.com

In view of the ongoing production capacity shortage in the semiconductor industry and the resultant price hike of chips, revenue of the top 10 IC design companies for 2Q21 reached US$29.8 billion, a 60.8% YoY increase, according to TrendForce‘s latest investigations. In particular, Taiwanese companies put up remarkable performances during this period, with both MediaTek and Novatek posting YoY growths of more than 95%. AMD, on the other hand, experienced a nearly 100% YoY revenue growth, the highest among the top 10.

TrendForce indicates that the ranking of the top five companies for 2Q21 remained unchanged from the previous quarter, although there were major changes in the 6th to 10th spots. More specifically, after finalizing its acquisition of Inphi, Marvell experienced a major revenue growth and leapfrogged Xilinx and Realtek in the rankings from 9th place in 1Q21 to 7th place in 2Q21.

Thanks to strong demand for major smartphone brands’ flagship and high-end 5G handsets, revenue leader Qualcomm’s processor and RF front-end businesses underwent remarkable growths, while its IoT business also benefitted from WFH and distance learning demands generated by the COVID-19 pandemic. Qualcomm’s revenue from its IoT business reached nearly US$1.4 billion, making IoT one of the major growth drivers for the company. For 2Q21, Qualcomm’s revenue reached US$6.47 billion, a 70.0% YoY increase. On the other hand, Nvidia’s revenues from gaming graphics cards and data center solutions each grew by 91.1% YoY and 46% YoY, respectively, in 2Q21. Strong demand from cryptocurrency miners for Nvidia’s high-end gaming graphics cards, along with the data center segment’s demand for Nvidia’s HPC products, propelled the company’s revenue for 2Q21 to US$5.84 billion, a 68.8% YoY growth, and secured the second place for Nvidia on the top 10 list.

Broadcom, which took third place on the top 10, attributed most of its revenue to wired connectivity and wireless products. Regarding wired connectivity products, the continued build-out of 5G base stations worldwide resulted in increasing demand for Broadcom’s high-speed Ethernet controller ICs, whereas for wireless products, the release of certain high-end 5G smartphones also created high demand for Broadcom’s Wi-Fi 6E chips. Similarly, Broadcom’s broadband and industrial solutions businesses both underwent double-digit growths in 2Q21, thereby driving the company’s revenue for 2Q21 to US$4.95 billion, a 19.2% YoY growth. Turning to AMD, the company’s revenue for 2Q21 reached US$3.85 billion, a staggering 99.3% YoY increase, owing to the following: first, the bullish gaming console market; second, massive earnings growths from enterprise, embedded, and semi-custom solutions; third, increased client adoption of AMD’s server CPUs (it should be noted that AMD’s server processor business grew by 183% YoY in 2Q21). AMD took fifth place in the top 10 list for 2Q21.

Regarding Taiwanese companies, MediaTek was able to sustain the momentum it gained in 1Q21 throughout 2Q21. MediaTek’s smartphone chip business, which generated the bulk of the company’s revenue, registered a 143% growth in 2Q21. At the same time, its revenues from other businesses also saw an overall double-digit growth. Hence, MediaTek posted a revenue of US$4.49 billion for 2Q21, a 98.8% YoY growth, and reached fourth place on the list. Finally, Novatek’s SoCs and display driver ICs both performed well in the market primarily due to its close partnerships with major foundries, including TSMC, UMC, and VIS. Revenue from display driver ICs, which had traditionally been Novatek’s primary revenue source, grew by 81% YoY in 2Q21.

Certain rumors in the end-devices markets indicate that demand will likely undergo a slowdown in 3Q21 and lead to decreased orders for certain components. However, given that foundries’ newly installed wafer capacities have yet to kick off mass production, the ongoing chip shortage is expected to persist for now. In addition, as some IC design companies’ client orders still remain unfulfilled, these companies’ revenues will likely experience further growths in 2H21, albeit to a relatively limited extent. It should also be pointed out that Marvell is expected to benefit from Inphi’s earnings for the next two quarters and increase its own revenue by more than 50% YoY in 2H21. Even so, Novatek’s sixth-place ranking is unlikely to be threatened by Marvell in the short run since Novatek will continue to benefit from the ongoing chip shortage and price hikes for the time being.

For more information on reports and market data from TrendForce’s Department of Semiconductor Research, please click here, or email Ms. Latte Chung from the Sales Department at lattechung@trendforce.com

The stay-at-home economy remains robust due to the ongoing COVID-19 pandemic, so the sales of gaming products such as game consoles and the demand for related components are being kept at a decent level, according to TrendForce’s latest investigations. However, the values of cryptocurrencies have plummeted in the past two months because of active interventions from many governments, with the graphics DRAM market entering into a bearish turn in 3Q21 as a result. While graphics DRAM prices in the spot market will likely show the most severe fluctuations, contract prices of graphics DRAM are expected to increase by 10-15% QoQ in 3Q21 since DRAM suppliers still prioritize the production of server DRAM over other product categories, and the vast majority of graphics DRAM supply is still cornered by major purchasers.

It should be pointed out that, given the highly volatile nature of the graphics DRAM market, it is relatively normal for graphics DRAM prices to reverse course or undergo a more drastic fluctuation compared with other mainstream DRAM products. As such, should the cryptocurrency market remain bearish, and manufacturers of smartphones or PCs reduce their upcoming production volumes in light of the ongoing pandemic and component supply issues, graphics DRAM prices are unlikely to experience further increase in 4Q21. Instead, TrendForce expects prices in 4Q21 to largely hold flat compared to the third quarter.

Sudden drop in ETH prices led to plummeting GDDR5 and GDDR6 spot prices

Recent observations on the spot trading of graphics DRAM products indicate that the changes in this market closely correlate to the changes in the value of ether (ETH) because graphics cards are the crucial tool for processing the mining algorithm of this cryptocurrency. ETH prices fell by more than 50% within a two-month span as a result of the latest measures enacted by regulatory agencies around the world to suppress the speculation of cryptocurrencies. Accordingly, cryptocurrency miners’ and investors’ interest in ETH has also diminished significantly. The plunging demand from cryptocurrency miners also means that a substantial number of graphics cards are being pushed into the second-hand market. TrendForce’s investigation shows that spot prices of graphics cards have fallen by about 20-60% over the past month or more. The differences in the magnitude of decline depends on brand and technology generation. Furthermore, the across-the-board decline in spot prices of graphics cards has also severely constrained the spot demand for graphics DRAM.

According to TrendForce’s understanding, even though spot prices are still higher than contract prices for GDDR6 chips, the difference is rapidly shrinking. This, in turn, will have an adverse effect on the general price trend of GDDR6 chips in the future. The trading is even more subdued for GDDR5 chips that are used in the earlier generations of graphics cards. Spot prices are now actually about 20% lower than contract prices for GDDR5 chips. The difference here indicates that there is a glut of older graphics cards, and the GDDR5 chips that are embedded in them are no longer in high demand.

Contract prices of graphics DRAM are expected to increase by nearly 15% for 3Q21 as graphics DRAM suppliers’ fulfillment rate remains relatively low

Regarding the contract market for graphics DRAM, the sell-side has considerable leverage in price negotiations as these suppliers prioritize the production of server DRAM ahead of other product categories. In the current ecosystem of discrete graphics cards, graphics DRAM buyers such as Nvidia are still opting for a business model based on bundle sales (that is, graphics card manufacturers that purchase Nvidia GPUs must also purchase graphics DRAM from Nvidia). Given that Nvidia and AMD have cornered the vast majority of graphics DRAM supply, notebook OEMs and small- and medium-sized manufacturers of computer components (such as motherboards) will find it difficult to procure sufficient graphics DRAM, while DRAM suppliers’ fulfillment rate for graphics DRAM chips remains relatively low. These aforementioned factors are responsible for not only the nearly 15% QoQ hike in the overall contract prices of graphics DRAM for 3Q21 (which is slightly higher than the corresponding price hikes in mainstream PC and server DRAM products for 3Q21), but also why spot prices of GDDR6 chips are about 10-15% higher than contract prices.

On the whole, prices in the graphics DRAM spot market, which is an extremely responsive market, have already begun to reflect the weakening demand from the end-product segment, particularly for graphics cards used in cryptocurrency mining. As the supply of second-hand graphics card increases, some graphics card manufacturers may thus kick off promotional price cuts to boost sales. In addition, buyers in the spot market may also begin anticipating even lower prices, and this anticipation will likely either lead to a massive decline in their graphics card demand or result in these buyers adopting a speculative attitude regarding graphics DRAM. TrendForce therefore believes that the gap between spot prices and contract prices of GDDR6 chips will begin to narrow in 3Q21.

For more information on reports and market data from TrendForce’s Department of Semiconductor Research, please click here, or email Ms. Latte Chung from the Sales Department at lattechung@trendforce.com

As the Taiwanese IPC (industrial PC) market suffered from deferred orders due to supply chain and logistical disruptions that took place in 1H20, total domestic IPC revenue for 1H20 reached NT$105.4 billion, a 4.7% YoY decrease, according to TrendForce’s latest investigations. However, given that the pandemic was gradually brought under control in 1H21, the market was able to benefit from strong demand from China’s 5G infrastructure rollout, as well as from expanded investments by Europe and the US in public infrastructures such as roads and railways aimed at facilitating an economic recovery. Hence, Taiwan’s IPC revenue for 1H21 reached NT$115.1 billion, a 9.2% YoY increase.

Regarding the financial performances of the top 10 IPC suppliers in Taiwan for 1H21, Ennoconn secured first place with a revenue of NT$42.95 billion, a 16.7% YoY increase. After its acquisition spree that began in 2010, Ennoconn is currently attempting to integrate its various subsidiaries’ technologies and resources in order to make headways in certain emerging technologies, including industrial automation, machine vision, HMI, and cloud services. Going forward, Ennoconn will cultivate its presence in the EV, smart healthcare, and smart retail sectors.

For 1H21, runner-up Advantech posted a revenue of NT$27.37 billion, an 8.2% YoY increase. While Advantech previously favored an acquisition-driven strategy, the company is now expanding into the smart healthcare, smart manufacturing, and smart city sectors primarily through technological partnerships and equity investments. Backed by its WISE-PaaS platform, Advantech continues to expand into the global markets by investing in overseas ISV (independent software vendors) and SI (systems integrators) in the aforementioned sectors.

DFI earned a third-place ranking in 1H21 with a revenue of NT$5.28 billion, a 25.2% YoY increase. After becoming part of the Qisda fleet in 2017, DFI subsequently went on to acquire telecom and information security solutions supplier AEWIN as well as industrial automation vendor Ace Pillar in 2019. These activities culminated in an annual revenue of NT$8.35 billion, an 18.8% YoY increase, for DFI in 2020. DFI currently specializes in smart manufacturing, smart healthcare, and intelligent transportation systems/infrastructures.

AI accelerator suppliers and IPC suppliers work in tandem to clearly define the AI value chain

IPC products have been widely used in AIoT and IIoT applications in recent years due to the proliferation of edge computing. As such, these products have also become the key determinant of how rapidly industries can adopt AI technologies such as machine vision. At the same time, IPC suppliers’ unique position in the mid-stream AI value chain means they are responsible for bridging the gap between upstream AI accelerator suppliers (including Intel, AMD, and Nvidia) and downstream ISV/SI.

With regards to the upstream AI value chain, Intel and AMD acquired independent FPGA suppliers Altera and Xilinx, respectively, in order to achieve more comprehensive heterogeneous computing competencies via horizontal integration. On the other hand, midstream IPC suppliers have been vertically integrating with downstream ISV/SI either independently or collectively through JVs, technological collaborations, strategic alliances, or M&A. For instance, Advantech and ADLINK are now operating on multi-strategy models as well as strategic collaboration models respectively, while Ennoconn and DFI are operating on M&A-oriented models.

On the whole, TrendForce expects that, as AI accelerator suppliers and IPC suppliers push integration forward in the AI value chain, not only will an increasing number of IPC products based on heterogeneous computing platforms be released to market, but emerging AI technologies such as machine vision will also see increased penetration in industrial automation applications. Hence, TrendForce expects annual machine vision revenue to reach US$86 billion in 2025.

In recent years, notebook computer (laptop) brands and processor suppliers alike have been actively adjusting their product strategies and business operations in response to behavioral shifts in the way consumers purchase and use computing devices. While notebook brands jostle for superiority in industrial design with improvements to their product appearances every year, competition in the processor industry has been even fiercer. TrendForce’s investigations indicate that the current competitive landscape in the processor industry consists of three developments, indicated below:

First, competition between AMD and Intel. Not only are both companies focused on expanding their respective ecosystems, but they have also been aiming to conquer the gaming market by releasing new products aimed at gamers this year. Apart from making headways in the PC processor market, AMD has introduced the AMD Advantage Design Framework.

AMD Advantage gaming notebooks are certified to meet standards of performance set by the company. AMD hopes that this certification system will allow it to generate a more consolidated gaming ecosystem while raising its brand equity. Intel, on the other hand, has been cultivating its presence in the creator and 5G notebook markets in an attempt to become the primary driver of digital transformation in the post-pandemic era.

Second, the Nvidia-Arm collaboration. This collaboration took place for the purpose of establishing an AI-enabled reference platform for notebook computers. More specifically, Arm’s CPU/NPU/GPU product stack delivers such wide-ranging AI solutions as real-time recognition, vibration detection, and keyword spotting. Following Arm’s successful foray into the PC segment, Nvidia will speed up its release of notebook products, including CPUs based on the Arm architecture.

Third, Qualcomm’s cross-sector ambitions. By architecting always connected notebooks* with Microsoft and Google, Qualcomm is now leveraging its advantages in 5G technology to prepare for upcoming competition with Intel in the 5G services market.

As the aforementioned companies’ presentations at Computex 2021 would suggest, not only do these processor suppliers possess their own competitive technological advantages, but they also share the common goal of upgrading their gaming competencies, including graphics cards, graphics technologies, and cooling performances.

Some of their current offerings aimed at the gaming market include the AMD RX 6000M, Nvidia RTX 3080 Ti/RTX 3070 Ti GPU, and Intel 11^th Gen Core H45. Interestingly, the AMD Advantage Design Framework, which certifies OEMs’ gaming notebooks based on the AMD platform, represents the company’s intention to challenge Nvidia’s dominance in the gaming market.

After Nvidia announced its US$40 billion acquisition of Arm last September, the partnership is expected to yield considerable technological synergies by way of the two companies’ AI collaboration. TrendForce believes that, in the long term, Arm Cortex CPUs based on the Armv9 architecture will allow Nvidia to break free from the dominance of Intel and AMD in the notebook CPU market. In particular, Nvidia will be able to cultivate its presence in the high-performance notebook market by combining its existing graphics technology with Arm Cortex CPUs.

Qualcomm’s main impetus for entering the notebook market can be attributed to the fact that the pandemic has brought about a new normal in which consumer adherence to notebook products has become increasingly strong. On the whole, Qualcomm’s cross-sector ambition appears to be on the cusp of victory, given the company’s preexisting 5G competencies and its experience in always-connected applications, advanced camera technologies, immersive audio/visual and display experiences, AI acceleration, and power efficiency for smartphones. As a case in point, Qualcomm is set to release Windows on Snapdragon notebooks as its own 5G *always connected PC platform.

It should be noted that Intel has also adopted MediaTek’s 5G chip technology in 5G connected notebooks featuring “Intel 5G Solution 5000”. On the other hand, Qualcomm is also developing mobile processors aimed at the entry-level always connected 4G/5G notebook market. TrendForce expects competition in the always connected market to generate a fresh wave of replacement demand in the mobile computing market.

*Always connected laptops (notebooks): notebooks that feature modem chips and have a constant internet connection much like smartphones. These notebooks can connect to the internet using 4G/5G networks without the need for Wi-Fi.

（Cover image source: Pixabay）