According to a report from Economic Daily News citing The Wallstreet Journal, Apple is rumored to be developing its own AI chips tailored for data centers, which could potentially give the world’s top smartphone seller a crucial advantage in the AI arms race. The report, quoting sources familiar with the matter, stated that Apple has been working closely with its chip manufacturing partner TSMC to design and produce these chips in the primary stage. However, it is still unclear whether the final version has been produced yet.

It is suggested that Apple’s server chips may focus on executing AI models, particularly in AI inference, rather than AI training, where Nvidia’s chips currently dominate.

Over the past decade, Apple has gradually become a major player in chip design for products like iPhone, iPad, Apple Watch, and Mac. The latest project involving Apple chips for data center servers, internally named “Project ACDC” (short for Apple Chips in Data Center), will integrate Apple’s IC design capabilities into the operation of clients’ servers, sources said.

The project has been in operation for several years, though the timetable for launching this server chip remains unclear. Apple is expected to unveil more new AI products and AI-related updates at its Worldwide Developers Conference (WWDC) in June.

An Apple spokesperson declined to comment on the reported developments.

According to reports from Wccftech on April 23rd, Apple is said to be working on a self-developed AI server processor using TSMC’s 3-nanometer process, with plans for mass production expected in the second half of 2025.

According to a Reuters, despite the U.S. expanding export controls on advanced artificial intelligence (AI) chips to China last year, Chinese universities and research institutions have recently acquired high-end AI chips from Nvidia through distributors.

Reviewing hundreds of bidding documents, Reuters found that since the U.S. expanded chip export controls on November 17 last year, ten Chinese entities have acquired Nvidia’s advanced chips embedded in server products produced by U.S. firms Supermicro, Dell, and Taiwanese company Gigabyte Technology.

Based on this Reuters report, bidding documents not reported from November 20 last year to February 28 this year show that Chinese institutions such as the Chinese Academy of Sciences, Shandong Artificial Intelligence Institute, Hubei Earthquake Administration, Shandong University, Southwest University, a technology investment company owned by the Heilongjiang Provincial Government, a state-owned aerospace research center, and a space science center have purchased these server products from distributors, which include some of Nvidia’s most advanced chips.

In response, a Nvidia spokesperson told Reuters that the products involved in these bids were exported before the ban was implemented in the United States. The spokesperson stated that the report does not imply that Nvidia or any of its partners violated export control regulations, and the proportion of these products in global sales is negligible. Nvidia complies with U.S. regulatory standards.

Both Supermicro and Dell stated that they would investigate and take action if any third-party illegal exports or re-exports are found. Gigabyte, the Taiwanese company mentioned in the report, told the Central News Agency that it has fully complied with relevant regulations since the chip ban took effect on November 17 last year, and has not shipped any restricted products to China. Gigabyte reiterated its strict adherence to relevant Taiwanese laws and international embargo regulations, stating that there has been no violation of any embargo regulations.

In 2023, the United States further restricted Chinese businesses from acquiring high-end AI chips. At that time, Nvidia responded by launching a China-specific version, the H20. TrendForce also presented relevant data for the Chinese market, indicating that Chinese CSP companies, including ByteDance, Baidu, Alibaba, and Tencent (BBAT), accounted for approximately 6.3% of high-end AI server shipments in 2023. Considering the ban and subsequent risks, it is estimated that the proportion in 2024 may be less than 4%.

(Photo credit: NVIDIA)

U.S. Commerce Secretary Gina Raimondo stated on January 26th that the U.S. government will propose that American cloud computing companies determine whether foreign entities are accessing U.S. data centers to train artificial intelligence models.

The proposed “know your customer” regulation was made available for public inspection on January 26th and is scheduled for publication on January 29th.

According to a report from Reuters, Raimondo stated during her interview that, “We can’t have non-state actors or China or folks who we don’t want accessing our cloud to train their models.”

“We use export controls on chips,” she noted. “Those chips are in American cloud data centers so we also have to think about closing down that avenue for potential malicious activity.”

Raimondo further claimed that, the United States is “trying as hard as we can to deny China the compute power that they want to train their own (AI) models, but what good is that if they go around that to use our cloud to train their models?”

Since the U.S. government introduced chip export controls to China last year, NVIDIA initially designed downgraded AI chips A800 and H800 for Chinese companies. However, new regulations in October of 2023 by the U.S. Department of Commerce brought A800, H800, L40S, and other chips under control.

Raimondo stated that the Commerce Department would not permit NVIDIA to export its most advanced and powerful AI chips, which could facilitate China in developing cutting-edge models.

In addition to the limitations on NVIDIA’s AI chips, the U.S. government has also imposed further restrictions on specific equipment. For example, ASML, a leading provider of semiconductor advanced lithography equipment, announced on January 1st, 2024, that it was partially revoking export licenses for its DUV equipment in relation to the U.S. government.

Read more

• [News] New Focus in the US-China Tech War! Can China Overtake with Silicon Photonics?
• [News] NVIDIA Rumored to Downgrade AI Chips for China Amid U.S. Restrictions, Year-End Mass Production Expected

(Photo credit: iStock)

According to the South Korean media The Korea Economic Daily’s report, Samsung Electronics has established a new business unit dedicated to developing next-generation chip processing technology. The aim is to secure a leading position in the field of AI chips and foundry services.

The report indicates that the recently formed research team at Samsung will be led by Hyun Sang-jin, who was promoted to the position of general manager on November 29. He has been assigned the responsibility of ensuring a competitive advantage against competitors like TSMC in the technology landscape.

The team will be placed under Samsung’s chip research center within its Device Solutions (DS) division, which oversees its semiconductor business, as mentioned in the report.

Reportedly, insiders claim that Samsung aims for the latest technology developed by the team to lead the industry for the next decade or two, similar to the gate-all-around (GAA) transistor technology introduced by Samsung last year.

Samsung has previously stated that compared to the previous generation process, the 3-nanometer GAA process can deliver a 30% improvement in performance, a 50% reduction in power consumption, and a 45% reduction in chip size. In the report, Samsung also claimed that it is more energy-efficient compared to FinFET technology, which is utilized by the TSMC’s 3-nanometer process.

Read more

• [News] IC Design Companies Seek Advanced Process Second Source, Overview of Competition Between TSMC and Samsung

(Photo credit: Samsung)

The fusion of AIGC with end-user devices is highlighting the importance of personalized user experiences, cost efficiency, and faster response times in generative AI applications. Major companies like Lenovo and Xiaomi are ramping up their efforts in the development of edge AI, extending the generative AI wave from the cloud to the edge and end-user devices.

On October 24th, Lenovo hosted its 9th Lenovo Tech World 2023, announcing deepening collaborations with companies like Microsoft, NVIDIA, Intel, AMD, and Qualcomm in the areas of smart devices, infrastructure, and solutions. At the event, Lenovo also unveiled its first AI-powered PC. This compact AI model, designed for end-user applications, offers features such as photo editing, intelligent video editing, document editing, and auto task-solving based on user thought patterns. 

Smartphone manufacturers are also significantly extending their efforts into edge AI. Xiaomi recently announced their first use of Qualcomm Snapdragon 8 Gen 3, significantly enhancing their ability to handle LLMs at the end-user level. Xiaomi has also embedded AI LLMs into their HyperOS system to enhance user experiences.

During the 2023 vivo Developer Conference on November 1st, vivo introduced their self-developed Blue Heart model, offering five products with parameters ranging from billions to trillions, covering various core scenarios. Major smartphone manufacturers like Huawei, OPPO, and Honor are also actively engaged in developing LLMs.

Speeding up Practical Use of AI Models in Business

While integrating AI models into end-user devices enhances user experiences and boosts the consumer electronics market, it is equally significant for advancing the practical use of AI models. As reported by Jiwei, Jian Luan, the head of the AI Lab Big Model Team from Xiaomi, explains that large AI models have gain attention because they effectively drive the production of large-scale informational content. This is made possible through users’ extensive data, tasks, and parameter of AI model training. The next step in achieving lightweight models, to ensure effective operation on end-user devices, will be the main focus of industry development.

In fact, generative AI’s combination with smart terminal has several advantages:

1. Personal data will not be uploaded to the cloud, reducing privacy and data security risks.
2. AI models can connect to end-user databases and personal information, potentially transforming general AI LLMs into personalized small models, offering personalized services to individual users.
3. By compressing AI LLMs and optimizing end-user hardware and software, edge AI can reduce operating costs, enhance response times, and increase service efficiency.

Users often used to complain about the lack of intelligence in AI devices, stating that AI systems would reset to a blank state after each interaction. This is a common issue with cloud-based LLMs. Handling such concerns at the end-user device level can simplify the process.

In other words, the expansion of generative AI from the cloud to the edge integrates AI technology with hardware devices like PCs and smartphones. This is becoming a major trend in the commercial application and development of large AI models. It has the potential to enhance or resolve challenges in AI development related to personalization, security and privacy risks, high computing costs, subpar performance, and limited interactivity, thereby accelerating the commercial use of AI models.

Integrated Chips for End-User Devices: CPU+GPU+NPU

The lightweight transformation and localization of AI LLMs rely on advancements in chip technology. Leading manufacturers like Qualcomm, Intel, NVIDIA, AMD, and others have been introducing products in this direction. Qualcomm’s Snapdragon X Elite, the first processor in the Snapdragon X series designed for PCs, integrates a dedicated Neural Processing Unit (NPU) capable of supporting large-scale language models with billions of parameters.

The Snapdragon 8 Gen 3 platform supports over 20 AI LLMs from companies like Microsoft, Meta, OpenAI, Baidu, and others. Intel’s latest Meteor Lake processor integrates an NPU in PC processors for the first time, combining NPU with the processor’s AI capabilities to improve the efficiency of AI functions in PCs. NVIDIA and AMD also plan to launch PC chips based on Arm architecture in 2025 to enter the edge AI market.

Kedar Kondap, Senior Vice President and General Manager of Compute and Gaming Business at Qualcomm, emphasizes the advantages of LLM localization. He envisions highly intelligent PCs that actively understand user thoughts, provide privacy protection, and offer immediate responses. He highlights that addressing these needs at the end-user level provides several advantages compared to solving them in the cloud, such as simplifying complex processes and offering enhanced user experiences.

To meet the increased demand for AI computing when extending LLMs from the cloud to the edge and end-user devices, the integration of CPU+GPU+NPU is expected to be the future of processor development. This underscores the significance of Chiplet technology.

Feng Wu, Chief Engineer of Signal Integrity and Power Integrity at Sanechips/ZTE, explains that by employing Die to Die and Fabric interconnects, it is possible to densely and efficiently connect more computing units, achieving large-scale chip-level hyperscale computing.

Additionally, by connecting the CPU, GPU, and NPU at high speeds in the same system, chip-level heterogeneity enhances data transfer rates, reduces data access power, increases data processing speed, and lowers storage access power to meet the parameter requirements of LLMs.

(Image: Qualcomm)