As the demand for AI GPUs increases, TSMC’s advanced packaging capacity for CoWoS is struggling to keep up. Recently, according to a report from Commercial Times, NVIDIA has reportedly turned to Intel for advanced packaging solutions.

According to industry sources cited by the same report, TSMC’s CoWoS-S and Intel’s Foveros packaging technologies are similar, allowing clients to turn to Intel and secures the capacity needed quickly.

Despite its current struggling on transformation, Intel has been gradually developing its ‘s foundry services. In addition to clients like Qualcomm and Microsoft, Intel’s advanced packaging has also attracted interest from companies like Cisco and AWS.

Under the IDM 2.0 strategy, Intel has opened up its wafer outsourcing and foundry services to customers, establishing an the independent IFS foundry service. Earlier this year, Intel secured a major USD 15 billon foundry order from Microsoft for the first system-level AI foundry service, which is expected to use the Intel 18A process.

The report from Commercial Times further suggested that Microsoft’s move is anticipated to reduce its heavy reliance on TSMC. The report also indicates that chip customers, including NVIDIA, have engaged with Intel. Intel’s flexible foundry strategy, which can provide advanced packaging, software, and chiplet services tailored to customer needs, has been well-received by chipmakers.

Sources cited by the same report reveal that the U.S. has begun allocating specialized funds to increase investments in the advanced packaging sector as well. This move could highlight the importance of advanced packaging as the next key area for global competition in production capacity.

In November last year, the U.S. Department of Commerce’s National Institute of Standards and Technology (NIST) released a report titled “National Advanced Packaging Manufacturing Program,” highlighting that advanced packaging technology is one of the key technologies in semiconductor manufacturing.

Additionally, the U.S. Department of Commerce plans to invest approximately USD 3 billion to advance the National Advanced Packaging Manufacturing Program. Intel, alongside Amkor, is another giant in local advanced packaging in the U.S.

The main focus of advanced packaging is on interconnect density, power efficiency, and scaling. From Foveros to hybrid bonding technology, Intel is gradually scaling down bumping pitch sizes, which allows for higher current loads and better thermal performance.

Furthermore, in May last year, Intel’s advanced packaging technology roadmap outlined plans to transition from traditional substrates to more advanced glass substrates.

Read more

• [News] Intel’s Earnings Fall Short, Cutting Over 15% of Workforce
• [News] Intel Hires Micron Executive to Lead Its Foundry Business

(Photo credit: Intel)

Amid the heating tech war between the U.S. and China, and the stringent sanctions imposed to prevent China from obtaining cutting-edge chips, it appears that China is still able to find its way out. According to a report by Tom’s Hardware, citing the New York Times, the latest tactic of China would be setting up new companies to trade advanced hardware and operating them until they are shut down.

Before that, it is understood that Chinese firms have been smuggling NVIDIA chips through some underground networks, which involve buyers, sellers and dispatchers, according to a previous report by the Wall Street Journals.

Now the country seems to find another option in order to evade the sanctions. According to the New York Times and Tom’s Hardware, buyers that include state-owned or affiliated companies, even sanctioned companies, are reportedly collaborating with the Chinese defense industry, as transactions ranging from a few hundreds of GPUs to a deal worth USD 103 million have been observed lately.

The new tactic, it is reported, would be to establish new companies to acquire advanced chips before facing U.S. sanctions. For instance, after Sugon, established under the strong promotion of the Chinese Academy of Sciences and focuses on fields like computing, storage, security and data center, was banned from obtaining NVIDIA chips due to its ties with the Chinese military, some former executives created a new company named Nettrix.

The reports further note that within six months, Nettrix became one of the largest Chinese manufacturers of AI servers, as tech giants including NVIDIA, Intel, and Microsoft have already begun doing business with it, all without violating any American laws. Given the company’s recent establishment, the U.S. likely hasn’t had the opportunity to thoroughly vet its background.

The reports suggest that the White House might significantly reduce Chinese backdoors in trade by ensuring that only licensed, white-listed buyers can legally procure these chips. However, many in the industry oppose increasingly stringent bans, arguing that they harm American companies more than they help.

Before the upcoming U.S. presidential election in November, the Biden authority is said to be considering a series of actions targeting semiconductors. The latest one includes new measures that might unilaterally impose restrictions on China as early as late August, preventing major memory manufacturers like Micron, SK hynix, and Samsung Electronics from selling high-bandwidth memory (HBM) to China.

• [News] US Reportedly Weighs Stricter Limits on AI Memory Access for China
• [News] China Reportedly Acquires Nvidia AI Chips Indirectly; Nvidia, Gigabyte Emphasize Compliance with U.S. Regulations

(Photo credit: Nettrix)

Apple’s latest technical document reveals that the two main AI models behind Apple Intelligence are trained using Google’s Tensor Processing Units (TPUs) instead of NVIDIA GPUs. According to a report from Commercial Times, this suggests that the demand for NVIDIA chips has outstripped supply, prompting some tech giants to seek alternatives.

Apple first introduced an AI technical document in June, briefly stating that its AI models were trained using TPUs. The latest technical document, which spans 47 pages, provides a detailed explanation of how Apple’s foundational models (AFM) and AFM servers are trained in Cloud TPU Clusters. This indicates that Apple rents cloud servers from cloud service providers to train its AI models.

In the document, Apple stated: “This system allows us to train the AFM models efficiently and scalably, including AFM-on-device, AFM-server, and larger models.”

Apple further mentioned that the on-device AFM models for iPhones and iPads are trained using a total of 2,048 TPUv5p chips, which are currently the most advanced TPU chips on the market. The AFM servers are trained using a total of 8,192 TPUv4 chips.

Google initially launched TPUs in 2015 for internal training use only and started offering TPU rental services to external clients in 2017. These TPUs are currently the most mature custom chips used for AI training. According to Google’s official website, the rental cost of their most advanced TPUs is approximately USD 2 per hour based on a three-year contract.

Though NVIDIA’s GPUs are currently dominating the high-end AI chip market, the enormous number of chips required for AI model training has led to a severe shortage. This is because major tech companies like OpenAI, Microsoft, Google, Meta, Oracle, and Tesla all use NVIDIA chips to develop their AI technologies.

Since the rise of ChatGPT at the end of 2022, which spurred the generative AI market, Silicon Valley tech giants have been racing to invest in AI research and development. In contrast, Apple has lagged behind its competitors and now has to intensify its efforts to bolster Apple Intelligence. On July 29th, Apple released a preview version of Apple Intelligence for certain devices.

Read more

• [News] NVIDIA’s H200 Order Delivered from Q3, Boosting Server Supply Chain with Strong Demand
• [News] US Government Reportedly Investigating Microsoft, NVIDIA, and OpenAI for AI Industry Monopolization

(Photo credit: NVIDIA)

According to Reuters, engineers at Amazon’s chip lab in Austin, Texas, recently tested highly confidential new servers. Per the Economic Times, the director of engineering at Amazon’s Annapurna Labs under AWS Rami Sinno revealed that these new servers feature Amazon’s AI chips, which can compete with NVIDIA’s chips.

It’s reported that Amazon is developing processors to reduce reliance on the costly NVIDIA chips, which will power some of Amazon’s AWS AI cloud services.

Amazon expects to use its self-developed chips to enable customers to perform complex calculations and process large amounts of data at a lower cost. The company’s competitors, Microsoft and Alphabet, are also pursuing similar efforts.

However, Amazon is a late starter in AI chip field, but a industrial leader in non-AI processing chip, whose main non-AI processing chip, Graviton, has been in development for nearly a decade and is now in its fourth generation. The other two AI chips, Trainium and Inferentia, are newer designs.

David Brown, AWS’s Vice President of Compute and Networking, stated that in some cases, the performance of these chips can be 40% to 50% higher compared to NVIDIA’s, and their cost is supposed to be about half of the same models of NVIDIA’s chips.

AWS accounts for nearly 20% of Amazon’s total revenue. The company’s revenue from January to March surged by 17% from the same period last year, reaching USD 25 billion. AWS controls about one-third of the cloud computing market, with Microsoft’s Azure comprising about 25%.

Amazon stated that it deployed 250,000 Graviton chips and 80,000 custom AI chips to handle the surge in platform activity during the recent Prime Day.

Read more

• [News] TSMC Reportedly Sees Strong 4nm Demand, with NVIDIA’s Order Up by 25%
• [News] Amazon Unveils New AWS-Designed Chips, Boosting Orders for TSMC and ALCHIP

(Photo credit: Amazon)

On July 1st, according to a report from Reuters, the French antitrust authority plans to file charges against NVIDIA, accusing the company of engaging in anti-competitive practices, making France the first country to take such action against NVIDIA.

The French competition regulator had raided NVIDIA’s local offices in September last year. At the time, they did not disclose the details of the investigation or the company involved, only stating it was related to the graphics card sector.

However, as per a previous report from Bloomberg, NVIDIA claimed that the French agency collected information from them regarding their business and competition in the graphics card and cloud service provider market as part of an ongoing inquiry into competition in those markets.

Sources cited by Reuters’ report indicated that last year’s raid was part of a broader investigation into cloud computing. With the surge in global chip demand following the advent of ChatGPT, NVIDIA, as the world’s largest manufacturer of AI and computer graphics cards, has naturally attracted close scrutiny from antitrust authorities in Europe and the United States.

NVIDIA previously disclosed in regulatory filings that both EU and French regulators had requested information about its graphics card products. The French antitrust authority has been actively investigating to understand NVIDIA’s key role in AI processors, its pricing policies, chip shortages, and the impact on prices.

Last Friday, the French authorities released a report on competition in generative AI, highlighting the risk of chip suppliers abusing their power. The report pointed out concerns about the chip industry’s heavy reliance on NVIDIA’s CUDA software for chip programming. Additionally, NVIDIA’s focus on investing in AI cloud service provider CoreWeave has also raised significant concerns among the authorities.

Reportedly, it is understood that companies violating French antitrust rules could face fines of up to 10% of their global annual revenue, though they can choose to make concessions to avoid penalties.

Moreover, the European Commission is currently gathering informal feedback to determine if NVIDIA has breached its antitrust rules, although it has not yet launched a formal investigation into anti-competitive behavior.

On the other hand, the New York Times reported on June 5th that the U.S. Department of Justice and the Federal Trade Commission (FTC) have reached an agreement, led by senior officials of both agencies, over the past week. The DOJ will investigate whether NVIDIA has violated antitrust laws, while the FTC will examine the conducts of OpenAI and Microsoft.

Read more

• [News] NVIDIA’s H200 Order Delivered from Q3, Boosting Server Supply Chain with Strong Demand
• [News] US Government Reportedly Investigating Microsoft, NVIDIA, and OpenAI for AI Industry Monopolization

(Photo credit: NVIDIA)