The fusion of AIGC with end-user devices is highlighting the importance of personalized user experiences, cost efficiency, and faster response times in generative AI applications. Major companies like Lenovo and Xiaomi are ramping up their efforts in the development of edge AI, extending the generative AI wave from the cloud to the edge and end-user devices.

On October 24th, Lenovo hosted its 9th Lenovo Tech World 2023, announcing deepening collaborations with companies like Microsoft, NVIDIA, Intel, AMD, and Qualcomm in the areas of smart devices, infrastructure, and solutions. At the event, Lenovo also unveiled its first AI-powered PC. This compact AI model, designed for end-user applications, offers features such as photo editing, intelligent video editing, document editing, and auto task-solving based on user thought patterns. 

Smartphone manufacturers are also significantly extending their efforts into edge AI. Xiaomi recently announced their first use of Qualcomm Snapdragon 8 Gen 3, significantly enhancing their ability to handle LLMs at the end-user level. Xiaomi has also embedded AI LLMs into their HyperOS system to enhance user experiences.

During the 2023 vivo Developer Conference on November 1st, vivo introduced their self-developed Blue Heart model, offering five products with parameters ranging from billions to trillions, covering various core scenarios. Major smartphone manufacturers like Huawei, OPPO, and Honor are also actively engaged in developing LLMs.

Speeding up Practical Use of AI Models in Business

While integrating AI models into end-user devices enhances user experiences and boosts the consumer electronics market, it is equally significant for advancing the practical use of AI models. As reported by Jiwei, Jian Luan, the head of the AI Lab Big Model Team from Xiaomi, explains that large AI models have gain attention because they effectively drive the production of large-scale informational content. This is made possible through users’ extensive data, tasks, and parameter of AI model training. The next step in achieving lightweight models, to ensure effective operation on end-user devices, will be the main focus of industry development.

In fact, generative AI’s combination with smart terminal has several advantages:

1. Personal data will not be uploaded to the cloud, reducing privacy and data security risks.
2. AI models can connect to end-user databases and personal information, potentially transforming general AI LLMs into personalized small models, offering personalized services to individual users.
3. By compressing AI LLMs and optimizing end-user hardware and software, edge AI can reduce operating costs, enhance response times, and increase service efficiency.

Users often used to complain about the lack of intelligence in AI devices, stating that AI systems would reset to a blank state after each interaction. This is a common issue with cloud-based LLMs. Handling such concerns at the end-user device level can simplify the process.

In other words, the expansion of generative AI from the cloud to the edge integrates AI technology with hardware devices like PCs and smartphones. This is becoming a major trend in the commercial application and development of large AI models. It has the potential to enhance or resolve challenges in AI development related to personalization, security and privacy risks, high computing costs, subpar performance, and limited interactivity, thereby accelerating the commercial use of AI models.

Integrated Chips for End-User Devices: CPU+GPU+NPU

The lightweight transformation and localization of AI LLMs rely on advancements in chip technology. Leading manufacturers like Qualcomm, Intel, NVIDIA, AMD, and others have been introducing products in this direction. Qualcomm’s Snapdragon X Elite, the first processor in the Snapdragon X series designed for PCs, integrates a dedicated Neural Processing Unit (NPU) capable of supporting large-scale language models with billions of parameters.

The Snapdragon 8 Gen 3 platform supports over 20 AI LLMs from companies like Microsoft, Meta, OpenAI, Baidu, and others. Intel’s latest Meteor Lake processor integrates an NPU in PC processors for the first time, combining NPU with the processor’s AI capabilities to improve the efficiency of AI functions in PCs. NVIDIA and AMD also plan to launch PC chips based on Arm architecture in 2025 to enter the edge AI market.

Kedar Kondap, Senior Vice President and General Manager of Compute and Gaming Business at Qualcomm, emphasizes the advantages of LLM localization. He envisions highly intelligent PCs that actively understand user thoughts, provide privacy protection, and offer immediate responses. He highlights that addressing these needs at the end-user level provides several advantages compared to solving them in the cloud, such as simplifying complex processes and offering enhanced user experiences.

To meet the increased demand for AI computing when extending LLMs from the cloud to the edge and end-user devices, the integration of CPU+GPU+NPU is expected to be the future of processor development. This underscores the significance of Chiplet technology.

Feng Wu, Chief Engineer of Signal Integrity and Power Integrity at Sanechips/ZTE, explains that by employing Die to Die and Fabric interconnects, it is possible to densely and efficiently connect more computing units, achieving large-scale chip-level hyperscale computing.

Additionally, by connecting the CPU, GPU, and NPU at high speeds in the same system, chip-level heterogeneity enhances data transfer rates, reduces data access power, increases data processing speed, and lowers storage access power to meet the parameter requirements of LLMs.

(Image: Qualcomm)

After several quarters of inventory depletion, the NAND market is gradually heating up, and there are concerns of a tighten supply in 2024. During an Investor Meeting held on the 7th, Phison Electronics Corporation, a supplier of NAND flash memory controllers and modules, announced that due to capacity limitations in NAND flash production, NAND supply has become constrained. When entering 4Q23, the company is already facing tighten supply for some of its products. Consequently, Phison plans to prepay its NAND flash suppliers to ensure a stable supply, as reported by CTEE.

Phison noted that its suppliers have been reducing production since the 4Q22, and it has accelerated since the 2Q23. Following three to four quarters of inventory depletion, the NAND market is gradually recovering, leading to a stabilization in NAND prices.

In fact, the strategy of module manufacturers is influenced by NAND flash suppliers. For instance, Samsung has been actively raising NAND prices. After the company initially raised NAND prices by 10% to 20% this quarter, it has decided to continue increasing prices by quarter in 2024. This strategic decision reflects Samsung’s determination to stabilize NAND prices with the aim of reversing the market’s direction in the first half of the upcoming year.

Notably, NAND chips and DRAM account for roughly half of Samsung’s memory chip sales. Simultaneously, while raising prices, Samsung continues to decrease production to control market supply, which, in turn, improves market stability and profitability.

TrendForce previously indicated that with NAND wafer prices leading the increase since August and suppliers adopting a firmer stance in negotiations, Q4 enterprise SSD contract prices are projected to rise by approximately 5~10%. On the client SSD front, as suppliers gain more bargaining power, both high-end and low-end products are expected to increase concurrently, with 4Q23 PC client SSD contract prices projected to rise by 8~13%.

TrendForce’s NAND Flash price analysis released today also highlighted that due to continuous shortages in recent wafer supply, the market has been experiencing rising prices under a shrinking volume. While spot quotations for NAND Flash packaged dies have been oscillating narrowly in quotations on account of the persistently constrained level of visibility in demand.
(Image: Samsung)

Explore more:

• [Insights] Weekly Price Update: DRAM Stalls, NAND Flash on the Rise
• [News] Samsung’s Announcement of a 20% Quarterly Price Increase for NAND Signals Promising Industry Trends
• Suppliers Scale Back Production, Q4 NAND Flash Contract Prices Projected to Climb 8~13%, Says TrendForce

In the spot market, DRAM prices are showing a slight divergence from the contract market. Spot demand has decreased, preventing further price increases for DRAM chips. Meanwhile, NAND Flash prices have been rising due to ongoing wafer shortage. However, spot quotations for NAND Flash packaged dies remain stable due to uncertain demand visibility.

DRAM Spot Market

The price trend of the spot market diverges slightly from that of the contract market. Recently, demand has dropped in the spot market. Even though DRAM suppliers and module houses have been passive in offering price concessions, there also has been no noticeable increase in demand. Furthermore, there is growing sentiment in the spot market that waiting for further developments is the best option. Hence, spot prices of DRAM chips have not been able to rise further. As for modules, their spot prices have experienced limited fluctuations because Kingston, as the leading module house, is unwilling to raise quotes due to its high inventory level. The average spot price of the mainstream chips (i.e., DDR4 1Gx8 2666MT/s) rose by 1.69% from US$1.600 last week to US$1.627 this week.

NAND Flash Spot Market

The spot market, due to continuous shortages in recent wafer supply, has been experiencing rising prices under a shrinking volume, while spot quotations for NAND Flash packaged dies have been oscillating narrowly in quotations on account of the persistently constrained level of visibility in demand. Follow ups on market prices can still be seen among products involved in production cuts of suppliers, whereas other products are fluctuating according to market demand. 512Gb TLC wafer spots have risen by 6.94% this week, arriving at US$2.464.

On November 7th, Intel held its “Intel Innovation Taipei 2023 Technology Forum”, with CEO Pat Gelsinger highlighting the healthy state of PC inventory. He also expressed optimism about the injection of several more years of innovative applications and evolution in PCs through AI. 

Intel Aims to Ship over One Hundred Million AI PC within the Next Two Years

Gelsinger expressed that the PC inventory has reached a healthy level, and he is optimistic about the future growth of AI PCs, which are equipped with AI processors or possess AI computing capabilities. He anticipates that AI will be a crucial turning point for the PC industry.

Additionally, Gelsinger stated that the server industry may have seemed uneventful in recent years, but with the accelerated development of AI, it has become more exciting. AI is becoming ubiquitous, transitioning from the training phase to the deployment phase, and various platforms will revolve around AI.

Gelsinger expressed his strong confidence in Intel’s position in the AI PC market, expecting to ship over one hundred million units within two years.

Intel’s Ambitious Expansion in Semiconductor Foundry Landscape

Intel is actively promoting its IDM 2.0 strategy, with expectations from the industry that the company, beyond its brand business, has advanced packaging capabilities to support semiconductor foundry operations. In the future, Intel is poised to compete with rivals such as TSMC and Samsung.

Gelsinger noted that some have viewed Intel’s plan of achieving five technical nodes in four years as “an ambitious endeavor.” However, he emphasized that Intel remains committed to its original goal of advancing five process nodes within four years.

The company’s foundry business has received positive responses from numerous potential customers, and while it may take three to four years for significant expansion, the advanced packaging aspect may only require two to three quarters to get on track.

This transformation marks a significant shift for the company, setting new standards in the industry. Intel is making steady progress in its four-year plan to advance five nodes, and Moore’s Law will continue to extend. The construction of Intel’s new factories is also ongoing.

According to Intel’s roadmap, Intel 7 and Intel 4 are already completed, Intel 3 is set for mass production in the latter half of this year, and Intel 20A and 18A are expected to enter mass production in the first and second halves of next year, respectively.

Regarding this roadmap, according to NIKKEI Asia’s report, Gelsinger also mentioned at the forum that for the 18A process, they currently have many test wafers in production. Additionally, the development of 18A has been completed, and it is progressing rapidly towards the production phase.

Read more

• [News] Intense Competition in Advancing Processes at the 2nm by Samsung, Intel, and TSMC

(Photo credit: Intel Newsroom)

During an earnings call in November 2023, Isao Matsumoto, President of ROHM Semiconductor, disclosed the company’s plans to the production of 8-inch SiC substrates at its second plant in Miyazaki, Japan, starting in 2024. This is the first time ROHM will produce SiC substrates in Japan.

Notably, the Miyazaki Plant No.2 Project is part of ROHM’s ongoing capacity expansion strategy. The company intends to invest between 170 and 220 billion yen in its SiC business from 2021 to 2025.

For SiC power semiconductor manufacturers like ROHM, it’s imperative to bolster their SiC substrate production capacity. This is driven by two key factors. Firstly, there has been a shortage of substrate materials, which has posed a challenge for the SiC power semiconductor industry. Secondly, the growing wave of electric vehicles has led to an increased demand for SiC power semiconductors.

To thrive in the SiC power semiconductor industry and capture a big market share, major companies are actively investing in expanding their production capacity. They often choose to form supply agreements with suppliers, set up their own production lines, or acquire related businesses.

ROHM previously established a pioneering position in the industry by producing SiC substrates at its plant in Nuremberg, Germany, following the acquisition of SiCrystal, a German SiC substrate manufacturer.

The Miyazaki plant no.2, scheduled to start production in 2024, was originally the Kunitomi plant of Solar Frontier, a subsidiary of Idemitsu Kosan. In July of this year, ROHM announced its acquisition of the assets of Solar Frontier’s former Kunitomi plant, a deal that concluded in October. It’s worth noting that this plant will become ROHM’s largest SiC power semiconductor production hub in Japan.

While pursuing acquisitions, ROHM is also actively expanding its in-house production capacity. According to information on ROHM’s official website, the company currently operates four SiC power semiconductor production plants in Japan, located at its Kyoto headquarters, Chikugo Plant in Fukuoka, Nagahama Plant in Fukuoka, and Miyazaki plant no.1.

Both acquisitions and in-house production strategies help ensure a reliable product supply. However, in the dynamic SiC industry, forward-thinking leaders are eager to secure additional production capacity in advance. Collaboration with established manufacturers is a common approach, and ROHM is no exception.

In June of this year, ROHM signed a long-term supply partnership agreement for SiC power devices with Vitesco. According to this agreement, the combined transactions between the two companies from 2024 to 2030 will exceed 130 billion yen.
(Image: ROHM)