The fusion of AIGC with end-user devices is highlighting the importance of personalized user experiences, cost efficiency, and faster response times in generative AI applications. Major companies like Lenovo and Xiaomi are ramping up their efforts in the development of edge AI, extending the generative AI wave from the cloud to the edge and end-user devices.

On October 24th, Lenovo hosted its 9th Lenovo Tech World 2023, announcing deepening collaborations with companies like Microsoft, NVIDIA, Intel, AMD, and Qualcomm in the areas of smart devices, infrastructure, and solutions. At the event, Lenovo also unveiled its first AI-powered PC. This compact AI model, designed for end-user applications, offers features such as photo editing, intelligent video editing, document editing, and auto task-solving based on user thought patterns. 

Smartphone manufacturers are also significantly extending their efforts into edge AI. Xiaomi recently announced their first use of Qualcomm Snapdragon 8 Gen 3, significantly enhancing their ability to handle LLMs at the end-user level. Xiaomi has also embedded AI LLMs into their HyperOS system to enhance user experiences.

During the 2023 vivo Developer Conference on November 1st, vivo introduced their self-developed Blue Heart model, offering five products with parameters ranging from billions to trillions, covering various core scenarios. Major smartphone manufacturers like Huawei, OPPO, and Honor are also actively engaged in developing LLMs.

Speeding up Practical Use of AI Models in Business

While integrating AI models into end-user devices enhances user experiences and boosts the consumer electronics market, it is equally significant for advancing the practical use of AI models. As reported by Jiwei, Jian Luan, the head of the AI Lab Big Model Team from Xiaomi, explains that large AI models have gain attention because they effectively drive the production of large-scale informational content. This is made possible through users’ extensive data, tasks, and parameter of AI model training. The next step in achieving lightweight models, to ensure effective operation on end-user devices, will be the main focus of industry development.

In fact, generative AI’s combination with smart terminal has several advantages:

1. Personal data will not be uploaded to the cloud, reducing privacy and data security risks.
2. AI models can connect to end-user databases and personal information, potentially transforming general AI LLMs into personalized small models, offering personalized services to individual users.
3. By compressing AI LLMs and optimizing end-user hardware and software, edge AI can reduce operating costs, enhance response times, and increase service efficiency.

Users often used to complain about the lack of intelligence in AI devices, stating that AI systems would reset to a blank state after each interaction. This is a common issue with cloud-based LLMs. Handling such concerns at the end-user device level can simplify the process.

In other words, the expansion of generative AI from the cloud to the edge integrates AI technology with hardware devices like PCs and smartphones. This is becoming a major trend in the commercial application and development of large AI models. It has the potential to enhance or resolve challenges in AI development related to personalization, security and privacy risks, high computing costs, subpar performance, and limited interactivity, thereby accelerating the commercial use of AI models.

Integrated Chips for End-User Devices: CPU+GPU+NPU

The lightweight transformation and localization of AI LLMs rely on advancements in chip technology. Leading manufacturers like Qualcomm, Intel, NVIDIA, AMD, and others have been introducing products in this direction. Qualcomm’s Snapdragon X Elite, the first processor in the Snapdragon X series designed for PCs, integrates a dedicated Neural Processing Unit (NPU) capable of supporting large-scale language models with billions of parameters.

The Snapdragon 8 Gen 3 platform supports over 20 AI LLMs from companies like Microsoft, Meta, OpenAI, Baidu, and others. Intel’s latest Meteor Lake processor integrates an NPU in PC processors for the first time, combining NPU with the processor’s AI capabilities to improve the efficiency of AI functions in PCs. NVIDIA and AMD also plan to launch PC chips based on Arm architecture in 2025 to enter the edge AI market.

Kedar Kondap, Senior Vice President and General Manager of Compute and Gaming Business at Qualcomm, emphasizes the advantages of LLM localization. He envisions highly intelligent PCs that actively understand user thoughts, provide privacy protection, and offer immediate responses. He highlights that addressing these needs at the end-user level provides several advantages compared to solving them in the cloud, such as simplifying complex processes and offering enhanced user experiences.

To meet the increased demand for AI computing when extending LLMs from the cloud to the edge and end-user devices, the integration of CPU+GPU+NPU is expected to be the future of processor development. This underscores the significance of Chiplet technology.

Feng Wu, Chief Engineer of Signal Integrity and Power Integrity at Sanechips/ZTE, explains that by employing Die to Die and Fabric interconnects, it is possible to densely and efficiently connect more computing units, achieving large-scale chip-level hyperscale computing.

Additionally, by connecting the CPU, GPU, and NPU at high speeds in the same system, chip-level heterogeneity enhances data transfer rates, reduces data access power, increases data processing speed, and lowers storage access power to meet the parameter requirements of LLMs.

(Image: Qualcomm)

Samsung’s upcoming flagship Galaxy S24 series, reportedly set to be unveiled in mid-January 2024, will feature Qualcomm’s latest Snapdragon 8 Gen 3 processor. This new series will highlight Samsung’s first-ever generative AI smartphone, with a sales target expected to increase by over 10% compared to the previous S23 model, reaching more than 34 million units.

As reported by “The Korea Daily,” Samsung is planning to unveil the Galaxy S24 series of new smartphones in mid-January 2024, making it the first major Android flagship to be launched next year. To ensure the successful market debut of the new devices, Samsung is expected to initiate the procurement process from its supply chain in the very near future.

Industry sources have revealed that Samsung’s internal estimates project smartphone sales for next year to reach approximately 253 million units and this figure is expected to be adjusted upwards in the near future according to Economic Daily.

In comparison, Apple’s projected total iPhone sales for the next year, according to a market analyst, are expected to be around 250 million units, making the estimates of these two major smartphone giants quite close.

It is reported that Samsung’s recent flagship devices in the S series have delivered subpar performance. Both the S21 and S22 models achieved sales figures of fewer than 30 million units, while the estimated sales for the S23 series stand at around 31 million units.

Notably, Samsung is currently collaborating with major players such as Google and Microsoft, and there is a strong likelihood that they will incorporate generative AI features like Google Bard or ChatGPT into the Galaxy S24.

This move aims to make the Galaxy S24 their first generative AI smartphone, serving as a prominent selling point for the new device and enhancing the operating system, thereby narrowing the gap with Apple.

Industry experts point out that while smartphone brands tend to estimate sales figures for new products before their launch, the smartphone market has experienced weakened demand in recent years.

Many brands have adopted a conservative approach to sales estimates, even controlling production capacity and marketing budgets to reduce risks. In this context, Samsung’s decision to set clear growth targets stands as a demonstration of their high confidence in the new product.

As for the supply chain, Largan Precision is the primary supplier of main camera components for the Galaxy S24, while GIS is responsible for supplying the in-display fingerprint module, and TXC Corporation provides quartz components.

(Photo credit: Samsung)

On November 6th, leading IC design company, MediaTek, introduced the Dimensity 9300, its latest flagship mobile System-on-Chip (SoC) featuring an innovative All Big Core design.

It incorporates groundbreaking technology to redefine flagship experiences in areas such as on-device AI, gaming, and imaging. The first smartphones to adopt MediaTek’s Dimensity 9300 chip are expected to hit the market by the end of 2023.

MediaTek’s next generation APU 790 AI processor is integrated into the Dimensity 9300 and designed to significantly improve generative AI performance and energy efficiency for faster and more secure edge computing. The APU 790 doubles the integer and floating-point operations performance, while reducing power consumption by 45%.

The APU 790 hardware includes a generative AI engine, enabling faster and more secure edge AI computations. It’s capable of accelerating operations on Transformer models, achieving processing speeds eight times faster than the previous generation and generating images within one second.

Moreover, in conjunction with large-scale language models, MediaTek has developed mixed-precision INT4 quantization technology, which when combined with the company’s NeuroPilot memory hardware compression, can more efficiently utilize memory bandwidth and significantly reduce memory requirements for large AI models.

The APU 790 provides support for NeuroPilot Fusion, which can continuously perform LoRA low-rank adaptation, and is capable of supporting large language models with 1B, 7B, and 13B parameters, with scalability up to 33B.

Additionally, MediaTek’s AI development platform, NeuroPilot, has built a rich AI ecosystem, supporting mainstream AI language models such as Meta LIama 2, Baidu ERNIE Bot, and Baichuan’s large language model, facilitating rapid and efficient deployment of multi-modal generative AI applications on edge devices, offering innovative user experiences in text, images, music, and more.

In terms of performance, MediaTek emphasizes that the Dimensity 9300 offers exceptional performance in terms of intelligence, efficiency, and low power consumption, all achieved through groundbreaking advanced technology. It sets the stage for a new flagship experience in generative AI, gaming, and imaging.

“The Dimensity 9300 is MediaTek’s most powerful flagship chip yet, bringing a huge boost in raw computing power to flagship smartphones with our groundbreaking All Big Core design,” said Joe Chen, President at MediaTek.

“This unique architecture, combined with our upgraded on-chip AI Processing Unit, will usher in a new era of generative AI applications as developers push the limits with edge AI and hybrid AI computing capabilities.”

MediaTek’s Over 40% Market Share in China Underscores the Vital Importance of the Chinese Market

Furthermore, MediaTek strategically chose China as the launch location for its latest flagship mobile chip. This decision reflects the long-standing partnerships with various Chinese smartphone brands and the fact that the Chinese market has consistently been a significant source of revenue for MediaTek.

For the third quarter of this year, mobile chips constituted 49% of the overall revenue, making a substantial contribution to the company’s earnings. Regarding geographical distribution, foreign securities firms have estimated that the Chinese market contributes to approximately 40% of MediaTek’s revenue, primarily driven by smartphone products.

Moreover, according to statements made by MediaTek’s Chairman Rick Tsai at the beginning of this year, following the introduction of the Dimensity 5G mobile chip series in 2019, MediaTek expanded its presence in the Chinese high-end smartphone chip market. Starting almost from scratch in 2021, it achieved a market share of 20% in 2022. The projection is that this trend will continue to rise this year.

In other words, the Chinese mobile chip market, particularly in the high-priced flagship mobile chip segment, remains a critical driving force for MediaTek. It presents a significant long-term competition challenge to Qualcomm, particularly as it enters the realm of generative AI business applications.

After Huawei’s breakthrough in 5G chipsets, enabling its return to the market with 5G smartphones, the sales of smartphones using Qualcomm’s 4G chipsets are bound to be affected, which, in turn, poses a challenge for Qualcomm. With the introduction of the 5G Generative AI mobile chipset Tianji 9300, the competition in the Chinese smartphone chipset market is expected to intensify.

(Photo credit: MediaTek)

During the Snapdragon Summit 2023 in October, Qualcomm made a big announcement by introducing the Snapdragon X Elite chip, built on TSMC’s cutting-edge 4nm process. Qualcomm’s claim is bold; they assert that the chip will deliver superior performance compared to Intel’s x86 architecture and Apple’s M2 chip. Simultaneously, towards the end of October, Apple revealed its own Arm-based M3 chip. Notably, semiconductor giants AMD and NVIDIA are rumored to be exploring the development of new PC processors on the Arm architecture. The future outlook suggests that Arm-based processors may gradually cut in the market share traditionally held by x86 architecture processors.

TrendForce’s Insights:

1. Qualcomm’s New PC Processor Poised to Expand the Arm-Based PC Processor Market

Following Intel’s September 2023 Meteor Lake processor release, Qualcomm introduced the Snapdragon X Elite, its latest Arm-based PC processor in October. This chip leverages TSMC’s advanced 4nm technology. Qualcomm is forging partnerships with Taiwanese heavyweights such as Quanta, Compal, Wistron, Acer, Asus, Realtek, Nuvoton and others, aiming to secure a foothold in the CPU market within the PC supply chain. This collaborative effort is expected to yield new PC products hitting the market in 2024.

As of 2023, Arm commands approximately 11% of the market share. Apple, a pioneer in PC chips built on the Arm architecture, has seen success with the release of three generations of processors since the 2020 debut of M1 chip.

With Qualcomm entering the Arm architecture group, AMD also intends to step into Arm architecture with the upcoming Phoenix processor, set for launch in 2025. There are also whispers of NVIDIA partnering with MediaTek to develop a chip processor featuring SoC+GPU capabilities on the Arm architecture, with an initial focus on ChromeBook market, where MediaTek boasts years of experience.

In the PC market, significant factors include processor development timelines, product performance, power efficiency, extended battery life, and compatibility with software and hardware. High-end CPUs and GPUs have become indispensable components for high-end computers. Currently, Intel leads the CPU market, with AMD following closely in the x86 landscape. As more companies delve into the development of Arm-based processors, there is potential for them to carve a share from the x86 market in the future.

2. Arm Architecture Processors Gain Momentum, Intel Focusing on AI Software Application Processor Development

Apple’s M-series chips demonstrate their efficiency and extended battery life. Microsoft recognizes the advantages of the Arm architecture and is dedicated to fostering a strategic collaboration for Windows on Arm. This collaboration involves integrating operating systems with processors to attract leading manufacturers to engage in Arm-based processor development, aligning with the growing demands for AI PCs.

In the realm of x86 architecture, Intel enjoys a dominant position in the CPU market. To tap into the opportunities arising from AI PCs, Intel has integrated AI acceleration engine features into its processors, introducing a new generation of AI application processors that combine CPU, GPU, and NPU (Neural Processing Unit) functionalities. This caters to the growing demand for generative AI applications in enterprise mobile PCs. Furthermore, Intel has joined forces with major laptop brands to launch new AI PCs, aiming to seize a substantial share of the market.

As momentum grows in Arm-based processor development, Intel maintains confidence that their immediate effect on the demand for x86 architecture processors will be restrained. It is unlikely to hinder Intel’s continuous advancement in developing new processors. Unlike competitors concentrating on Arm architecture processor development, Intel places a stronger focus on AI software applications and the market opportunities arising from its partnership with the Microsoft platform.

Explore more:

• [News] Qualcomm Challenges Intel & AMD in AI PC Chip Race
• [News] Intel, Samsung, TSMC Race in Cutting-Edge Processes

In the fourth quarter, Qualcomm reported a year-on-year revenue decline of 24%, amounting to $8.67 billion for the period ending on September 24. Nevertheless, this figure exceeded the market’s projected $8.51 billion. Akash Palkhiwala, CFO of Qualcomm, stated that the smartphone market is showing signs of stabilization, and Qualcomm’s chip sales are on the path to recovery.

CEO Cristiano Amon further elaborated that most smartphone companies have successfully bring up their inventory and are now placing new orders. He expressed, “we are cautiously optimistic as we go forward with that stabilizing and the normalization of Android channel inventory.”

According to reports from China Times, Huawei, once a former client of Qualcomm, has ventured into designing and producing its smartphone chips, thus emerging as a new competitor for Qualcomm. Amon emphasized that he thought Huawei’s return to the market will affect very small to Qualcomm’s relationships with other Chinese smartphone manufacturers. CFO Palkhiwala indicated that sales to Chinese smartphone clients are expected to grow by 35% compared to the previous quarter.

Notably, Qualcomm’s prominent client, Samsung Electronics, continues to rely on Qualcomm chips for the majority of its recent devices. Amon confirmed that Qualcomm will continue to dominate the chips used in Samsung’s upcoming S24 series of smartphones. Financial analyst Logan Purk from Edward Jones highlighted that the resurgence in demand for Android smartphones is closely tied to the significant growth in demand from Chinese OEM manufacturers. This may be a key factor contributing to Qualcomm’s consistent performance and improved outlook for the next quarter.

TrendForce, focusing on the smartphone industry, analyzed that the completion of inventory adjustments by leading process manufacturers is set to stimulate activity for upstream foundries. However, when it comes to the overall recovery of the smartphone market, there are still many uncertainties present, such as ongoing conflicts, the economic revival of major powers, and geopolitical factors. Even though the market has shifted from an extremely pessimistic outlook, there is still no clear signal supporting an optimistic and upward trend. Consequently, a slight growth of approximately 3% over the course of the year is expected.
(Image: Qualcomm)