Apple’s latest technical document reveals that the two main AI models behind Apple Intelligence are trained using Google’s Tensor Processing Units (TPUs) instead of NVIDIA GPUs. According to a report from Commercial Times, this suggests that the demand for NVIDIA chips has outstripped supply, prompting some tech giants to seek alternatives.

Apple first introduced an AI technical document in June, briefly stating that its AI models were trained using TPUs. The latest technical document, which spans 47 pages, provides a detailed explanation of how Apple’s foundational models (AFM) and AFM servers are trained in Cloud TPU Clusters. This indicates that Apple rents cloud servers from cloud service providers to train its AI models.

In the document, Apple stated: “This system allows us to train the AFM models efficiently and scalably, including AFM-on-device, AFM-server, and larger models.”

Apple further mentioned that the on-device AFM models for iPhones and iPads are trained using a total of 2,048 TPUv5p chips, which are currently the most advanced TPU chips on the market. The AFM servers are trained using a total of 8,192 TPUv4 chips.

Google initially launched TPUs in 2015 for internal training use only and started offering TPU rental services to external clients in 2017. These TPUs are currently the most mature custom chips used for AI training. According to Google’s official website, the rental cost of their most advanced TPUs is approximately USD 2 per hour based on a three-year contract.

Though NVIDIA’s GPUs are currently dominating the high-end AI chip market, the enormous number of chips required for AI model training has led to a severe shortage. This is because major tech companies like OpenAI, Microsoft, Google, Meta, Oracle, and Tesla all use NVIDIA chips to develop their AI technologies.

Since the rise of ChatGPT at the end of 2022, which spurred the generative AI market, Silicon Valley tech giants have been racing to invest in AI research and development. In contrast, Apple has lagged behind its competitors and now has to intensify its efforts to bolster Apple Intelligence. On July 29th, Apple released a preview version of Apple Intelligence for certain devices.

Read more

• [News] NVIDIA’s H200 Order Delivered from Q3, Boosting Server Supply Chain with Strong Demand
• [News] US Government Reportedly Investigating Microsoft, NVIDIA, and OpenAI for AI Industry Monopolization

(Photo credit: NVIDIA)

According to Reuters, engineers at Amazon’s chip lab in Austin, Texas, recently tested highly confidential new servers. Per the Economic Times, the director of engineering at Amazon’s Annapurna Labs under AWS Rami Sinno revealed that these new servers feature Amazon’s AI chips, which can compete with NVIDIA’s chips.

It’s reported that Amazon is developing processors to reduce reliance on the costly NVIDIA chips, which will power some of Amazon’s AWS AI cloud services.

Amazon expects to use its self-developed chips to enable customers to perform complex calculations and process large amounts of data at a lower cost. The company’s competitors, Microsoft and Alphabet, are also pursuing similar efforts.

However, Amazon is a late starter in AI chip field, but a industrial leader in non-AI processing chip, whose main non-AI processing chip, Graviton, has been in development for nearly a decade and is now in its fourth generation. The other two AI chips, Trainium and Inferentia, are newer designs.

David Brown, AWS’s Vice President of Compute and Networking, stated that in some cases, the performance of these chips can be 40% to 50% higher compared to NVIDIA’s, and their cost is supposed to be about half of the same models of NVIDIA’s chips.

AWS accounts for nearly 20% of Amazon’s total revenue. The company’s revenue from January to March surged by 17% from the same period last year, reaching USD 25 billion. AWS controls about one-third of the cloud computing market, with Microsoft’s Azure comprising about 25%.

Amazon stated that it deployed 250,000 Graviton chips and 80,000 custom AI chips to handle the surge in platform activity during the recent Prime Day.

Read more

• [News] TSMC Reportedly Sees Strong 4nm Demand, with NVIDIA’s Order Up by 25%
• [News] Amazon Unveils New AWS-Designed Chips, Boosting Orders for TSMC and ALCHIP

(Photo credit: Amazon)

According to a report from Commercial Times, TSMC’s advanced processes continue to experience strong demand, with major tech companies such as Apple, Qualcomm, NVIDIA, and AMD nearly fully booking TSMC’s 3nm capacity, while the order visibility extends into 2025.

As per the same report, the supply chain of the foundry giant’s 3nm process is also expected to continue the momentum, benefiting companies like Gudeng Precision Industrial (Gudeng), KINIK Company (KINIK), Solar Applied Materials Technology Corp (Solar Applied Materials), YEEDEX and GreenFiltec.

Industry sources cited by the same report indicate that TSMC’s wafer prices for advanced process are expected to see double-digit percentage increases by 2025. Chairman C.C. Wei recently mentioned that TSMC’s wafer pricing is strategic, as values regarding products may differ. The progress in pricing negotiations with customers is “so far so good,” and TSMC is confident about achieving a balance between price and capacity.

A report from Wccftech also suggested that Apple’s upcoming A18 Pro SoC and NVIDIA’s next-generation “Rubin” architecture will both use the 3nm process. Following the widespread adoption of TSMC’s 3nm process, the 3nm supply chain has attracted significant market attention, with expectations for operational boosts in the coming year.

In the semiconductor industry, per the sources cited by Commercial Times, Gudeng has captured about 70% of the market share for EUV POD. As EUV lithography has become a standard in advanced processes, TSMC has been adopting EUV technology starting from the 7nm process node and increasing its usage in later generations. The number of EUV layers used per wafer has been growing, which bodes well for Gudeng as it continues to benefit from this trend.

In the semiconductor sputtering target materials sector, TSMC has traditionally relied on products from international suppliers. However, with a push towards domestic equipment and consumables, Solar Applied Materials entered TSMC’s advanced 7nm process supply chain last year and has now also become a supplier for the 3nm process. Solar Applied Materials’ revenue from semiconductor targets was 4% last year and is expected to reach 10% this year, with a target of 20% by 2026, according to the report.

YEEDEX specializes in supplying components for the front-end processes, such as precision vaccum chucks for EUV equipment. As the industry moves to 3nm processes, wafer thinness increases, making precision vaccum chucks crucial for improving yield rates.

GreenFiltec specializes in extractable chemical filters and AMC Micro Pollution Control services. Through innovative material research and development for filters, GreenFiltec prevents invisible gas molecules from settling in the air. These filters are key consumables for cost control and yield improvement. In the domestic advanced process market, GreenFiltec has captured over 50% of the market share, and its operations are expected to benefit similarly next year, the report noted.

KINIK produces reclaimed wafers and diamond discs. Its high-spec diamond disc products have gained continuous and expanded adoption from major clients, achieving over 70% market share in the 3nm segment. The sources cited by Commercial Times expect that as major clients’ 3nm production capacity reaches full utilization, KINIK’s performance will significantly improve starting next year.

Read more

• [News] AMD Reportedly Eyes Mass Production in 2025 for Zen 6 Architecture with TSMC’s N3E Process
• [News] Apple Not Adopting TSMC 2nm for iPhone 17? A19 Pro Chip Rumored to Use N3P Process

(Photo credit: TSMC)

According to a report from Wccftech, it’s indicated that with soaring market demand, the shipment volume of NVIDIA’s Blackwell architecture GB200 AI servers has also significantly increased.

As NVIDIA claims, the Blackwell series is expected to be its most successful product. Industry sources cited by Wccftech indicate that NVIDIA’s latest GB200 AI servers are drawing significant orders, with strong demand projected to continue beyond 2025. This ongoing demand is enabling NVIDIA to secure additional orders as its newest AI products remain dominant.

The increasing demand for NVIDIA GB200 AI servers has led to revenue performances of Taiwanese suppliers such as Quanta, Foxconn, and Wistron exceeding expectations. Reportedly, NVIDIA is expected to ship 60,000 to 70,000 servers equipped with GB200 AI server. Each server is estimated to cost between USD 2 million and 3 million, resulting in approximately USD 210 billion in annual revenue from the Blackwell servers alone.

NVIDIA’s GB200 AI chip servers, available in NVL72 and NVL36 specifications, have seen greater preference for the less powerful models due to the growing number of AI startups choosing the more financially feasible NVL36 servers.

Notably, according to the industry sources previously cited in a report from Economic Daily News, TSMC is gearing up to start production of NVIDIA’s latest Blackwell platform architecture graphics processors (GPU) on the 4nm process.

The same report further cited sources, revealing that international giants such as Amazon, Dell, Google, Meta, and Microsoft will adopt the NVIDIA Blackwell architecture GPU for AI servers. As demand exceeds expectations,NVIDIA is prompted to increase its orders with TSMC by approximately 25%.

Read more

• [News] NVIDIA Reportedly Developing Tailored Version of the Blackwell Series for Chinese Market
• [News] TSMC Reportedly Sees Strong 4nm Demand, with NVIDIA’s Order Up by 25%

(Photo credit: NVIDIA)

As SK hynix and Samsung are releasing their financial results on July 25^th and July 31st, respectively, their progress on HBM3 and HBM3e have also been brought into spotlight. Earlier this week, Samsung is said to eventually passed NVIDIA’s qualification tests for its HBM3 chips. While the Big Three in the memory sector are now almost on the same page, the war between HBM3/ HBM3e is expected to intensify in the second half of 2024.

Samsung Takes a Big Leap

According to reports from Reuters and the Korea Economic Daily, Samsung’s HBM3 chips have been cleared by NVIDIA, which will initially be used exclusively in the AI giant’s H20, a less advanced GPU tailored for the Chinese market. Citing sources familiar with the matter, the reports note that Samsung may begin supplying HBM3 to NVIDIA as early as August.

However, as the U.S. is reportedly considering to implement new trade sanctions on China in October, looking to further limit China’s access to advanced AI chip technology, NVIDIA’s HGX-H20 AI GPUs might face a sales ban. Whether and to what extent would Samsung’s momentum be impacted remains to be seen.

SK hynix Eyes HBM3e to Account > 50% of Total HBM Shipments

SK hynix, as the current HBM market leader, has expressed its optimism in securing the throne on HBM3. According to a report by Business Korea, citing Kim Woo-hyun, vice president and chief financial officer of SK hynix, the company significantly expanded its HBM3e shipments in the second quarter as demand surged.

Moreover, SK hynix reportedly expects its HBM3e shipments to surpass those of HBM3 in the third quarter, with HBM3e accounting for more than half of the total HBM shipments in 2024.

SK hynix started mass production of the 8-layer HBM3e for NVIDIA in March, and now it is also confident about the progress on the 12-layer HBM3e. According to Business Korea, the company expects to begin supplying 12-layer HBM3e products to its customers in the fourth quarter. In addition, it projects the supply of 12-layer products to surpass that of 8-layer products in the first half of 2025.

Micron Expands at Full Throttle

Micron, on the other hand, has reportedly started mass production of 8-layer HBM3e in February, according to a previous report from Korea Joongang Daily. The company is also reportedly planning to complete preparations for mass production of 12-layer HBM3e in the second half and supply it to major customers like NVIDIA in 2025.

Targeting to achieve a 20% to 25% market share in HBM by 2025, Micron is said to be building a pilot production line for HBM in the U.S. and is considering producing HBM in Malaysia for the first time to capture more demand from the AI boom, a report by Nikkei notes. Micron’s largest HBM production facility is located in Taichung, Taiwan, where expansion efforts are also underway.

Earlier in May, a report from a Japanese media outlet The Daily Industrial News also indicated that Micron planned to build a new DRAM plant in Hiroshima, with construction scheduled to begin in early 2026 and aiming for completion of plant buildings and first tool-in by the end of 2027.

TrendForce’s latest report on the memory industry reveals that DRAM revenue is expected to see significant increases of 75% in 2024, driven by the rise of high-value products like HBM. As the market keeps booming, would Samsung come from behind and take the lead in the HBM3e battle ground? Or would SK hynix defend its throne? The progress of 12-layer HBM3e may be a key factor to watch.

Read more

• [News] SK hynix Rumored to Team up with Amkor to Secure HBM Advantage over Samsung
• [News] Samsung’s HBM3e Rumored to be Certified by NVIDIA, Boosting DDR5 Price Increases in Q3

(Photo credit: Samsung)