Apple’s latest technical document reveals that the two main AI models behind Apple Intelligence are trained using Google’s Tensor Processing Units (TPUs) instead of NVIDIA GPUs. According to a report from Commercial Times, this suggests that the demand for NVIDIA chips has outstripped supply, prompting some tech giants to seek alternatives.

Apple first introduced an AI technical document in June, briefly stating that its AI models were trained using TPUs. The latest technical document, which spans 47 pages, provides a detailed explanation of how Apple’s foundational models (AFM) and AFM servers are trained in Cloud TPU Clusters. This indicates that Apple rents cloud servers from cloud service providers to train its AI models.

In the document, Apple stated: “This system allows us to train the AFM models efficiently and scalably, including AFM-on-device, AFM-server, and larger models.”

Apple further mentioned that the on-device AFM models for iPhones and iPads are trained using a total of 2,048 TPUv5p chips, which are currently the most advanced TPU chips on the market. The AFM servers are trained using a total of 8,192 TPUv4 chips.

Google initially launched TPUs in 2015 for internal training use only and started offering TPU rental services to external clients in 2017. These TPUs are currently the most mature custom chips used for AI training. According to Google’s official website, the rental cost of their most advanced TPUs is approximately USD 2 per hour based on a three-year contract.

Though NVIDIA’s GPUs are currently dominating the high-end AI chip market, the enormous number of chips required for AI model training has led to a severe shortage. This is because major tech companies like OpenAI, Microsoft, Google, Meta, Oracle, and Tesla all use NVIDIA chips to develop their AI technologies.

Since the rise of ChatGPT at the end of 2022, which spurred the generative AI market, Silicon Valley tech giants have been racing to invest in AI research and development. In contrast, Apple has lagged behind its competitors and now has to intensify its efforts to bolster Apple Intelligence. On July 29th, Apple released a preview version of Apple Intelligence for certain devices.

Read more

• [News] NVIDIA’s H200 Order Delivered from Q3, Boosting Server Supply Chain with Strong Demand
• [News] US Government Reportedly Investigating Microsoft, NVIDIA, and OpenAI for AI Industry Monopolization

(Photo credit: NVIDIA)

MediaTek is making further strides in AI applications, focusing on the integration of smartphones and AR/MR devices. According to a report from the Economic Daily News, they look to capitalize on the significant business opportunities presented by 3D imaging combined with generative AI for immersive experiences. The sources cited in the same report indicated that MediaTek has formed an alliance with Meta, leveraging MediaTek’s Dimensity series smartphone chips as a platform alongside Meta’s Quest devices to target this market.

Previously, tech giants like Apple and Qualcomm have also recognized the potential of 3D imaging combined with generative AI for immersive experiences, and thus have been actively developing their strategies. While Apple builds its ecosystem through iPhones and Vision Pro headsets, Qualcomm has strengthened its collaboration with Google. With Meta and MediaTek entering the fray, the competition in the 3D imaging market will be further intensified.

Industry sources cited by the same report indicated that the generative AI business opportunity is set to explode, as cloud service providers (CSPs) are actively building AI servers, engaging in a “computing power war.” This hints that future AI market demand is likely to extend from the cloud to edge devices, thereby expanding AI applications to smartphones and AR/MR-related end-user devices.

Apple, which just showcased its upcoming products at WWDC, has launched the MR device Vision Pro previously, and is reportedly looking to expand its AI technology layout further. This includes integrating AI capabilities into iOS 18 in the iPhone 16 series, which will be released in September.

Meanwhile, per the same report, it’s expected that the iPhone 16 series will significantly enhance 3D photography features and improve integration with the Vision Pro. This indicates that 3D imaging will become a new application frontier in Apple’s AI strategy.

Non-Apple camps are also sensing these trends and opportunities. Qualcomm is reportedly teaming up with Google to integrate related systems in smartphones and wearable devices.

It is reported that Google has already strengthened its hardware development team in Taiwan. By doing so, the tech giant is possibly aiming to collaborate directly with major semiconductor companies like TSMC to develop 3D imaging applications. To bolster its future AI strategy, Qualcomm will collaborate with Google to integrate 3D imaging platforms into smartphones.

Industry sources cited by the report suggested that beyond language models, imaging is one of the AI applications that provide tangible experiences for users. Currently, AI applications in imaging are mostly focused on photo editing and cartoonization. However, with the improvement of camera functionalities in mobile devices, AI will begin to be integrated into 3D imaging.

As a result, major mobile platform providers will not only emphasize AI processing chips in their future hardware specifications but also upgrade imaging hardware specifications. This is expected to become a new battleground, potentially sparking a new wave of AI-driven smartphone upgrades.

Read more

• [News] Four Semiconductor Applicants for Taiwan’s Chip Act Tax Incentives – TSMC and MediaTek Expected to Benefit
• [News] MediaTek Announces 4nm Dimensity 7300 Series Chips, Supporting Foldable Devices

(Photo credit: MediaTek)

Just as generative AI is revolutionizing industries worldwide today by creating new opportunities, the concept of the Metaverse in 2021 was similarly embraced by technology giants as a strategic goal and vision for the future of the entire tech sector.

Microsoft, for instance, targeted the corporate segment of the Metaverse, showcasing the use of its mixed reality (MR) device, HoloLens 2, in manufacturing operations. NVIDIA introduced the Omniverse platform for image simulation, thereby facilitating the development of virtual environments within the Metaverse. There were also rumors about Google and Apple launching new virtual reality (VR) head-mounted devices. Perhaps most notably, Facebook’s name change to Meta was a clear indication of its commitment to this emerging field.

As Hype Fades, More Hardware and Content Are Needed to Strengthen the Foundation of the Metavers

Despite initial market optimism, the reality was that wearable technology had not reached maturity, and the quality of virtual content experiences fell short of expectations. As a result, there was insufficient momentum to drive the Metaverse forward in subsequent market developments. Many tech companies established departments dedicated to the Metaverse, but due to lackluster results and issues with resource allocation, these departments often faced workforce reductions, downsizing, or even complete dissolution.

Declining enthusiasm for the Metaverse primarily stems not from a flaw in the idea of blending virtual and real worlds, but from the grandiosity of its concept. The essence of Industry 4.0, after all, revolves around enhancing production efficiency through the data-driven integration of physical and digital realms.

This is a proven approach. Nevertheless, the challenge with the Metaverse lies in its ambitious scale. Without adequate software and hardware support, efforts to expand and implement it often fall short, yielding minimal benefits and, thus, diminishing its commercial appeal.

Essentially, the widespread adoption of technologies like head-mounted devices and a rich content library are vital for industry growth. In response, companies that develop VR and augment reality (AR) in recent years have pivoted their focus from the broader environmental framework towards improving wearable devices and creating engaging content. In doing so, they aim to boost the practical value of adopting VR and AR.

From Virtual Interaction to Spatial Computing, the Scope of Applications for Head-mounted Devices Continues to Expand

In 2023, according to TrendForce’s analysis, Meta’s Quest series dominated the global VR and MR device market, securing nearly 70% of total device shipments. This significant market share places Meta at the forefront, with Sony’s PS VR series ranking second, followed by other manufacturers like PICO and HTC. Entering the fray in 2024, Apple introduced its Vision Pro, which is expected to claim a 6% share of the global market.

Meta’s latest offering, the Quest 3, has adopted pancake lenses that enhance image clarity while slimming down the device’s profile. It is powered by the Qualcomm Snapdragon XR2 Gen 2, a (SoC) tailored for head-mounted devices that significantly boosts GPU and AI processing capabilities.

The Quest 3 marks a pivotal shift for Meta from VR to MR. Equipped with dual front-facing RGB cameras and advanced features like depth projection and room mapping, the Quest 3, alongside the higher-end Quest Pro, supports a range of MR applications. Additionally, the tracking capabilities of the Quest 3 are augmented by computer vision and machine learning technologies. With Meta’s ongoing collaboration with LG on new product development, the focus is now on extended reality (XR) applications linked with the television ecosystem.

Apple’s Vision Pro, which was launched in February 2024, has reignited market interest in VR.

This device fills a previously unaddressed gap in Apple’s portfolio by offering a VR head-mounted device that integrates seamlessly with iPhones, iPads, and other devices within Apple’s ecosystem, thereby enabling functions like image and video projection onto larger screens. The introduction of the Vision Pro brought the concept of spatial computing into the limelight, enabling users to interact with virtual objects in a natural and intuitive way and thus infusing fresh perspectives into the industry.

Moreover, at CES 2024, Sony unveiled an XR head-mounted device dubbed a “spatial content creation system.” Like the Apple Vision Pro, this device leverages the advantages of spatial computing. It’s designed as a commercial tool for developing 3D content, offering users precise and intuitive control over virtual objects, thereby simplifying the process of creating 3D models.

From Taiwan, ASUS has recently introduced its first AR glasses, the AirVision M1. These glasses are designed to function as a secondary screen, ideal for use outdoors or in situations where extra screens are necessary at home.

Taiwan-based Companies Expand into the Supply Chain for Headsets, Focusing on Optics, Chips, and Assembly

TrendForce analyst P. K. Tseng said that a critical aspect of the transformation for VR head-mounted devices is the increasing need for key components that are lighter and more compact, particularly pancake lenses, which are gaining importance due to their contribution to volume reduction.

However, the technological complexity and higher cost of manufacturing these advanced optical components mean that suppliers, such as GSEO and Young Optics, are relatively limited. This presents a blue ocean market opportunity, likely attracting more manufacturers to develop pancake lens components.

Furthermore, the trend is expected to drive demand for smaller-sized panels. While mainstream LCD panels continue to be widely used, the advent of devices like the Apple Vision Pro is anticipated to increase the adoption rate of Micro OLED panels.

Additionally, as standalone virtual devices become more mainstream in product design, and as the need for processing large volumes of image and sensor data independently by SoCs grows, demand will rise for dedicated chips used in VR and AR devices. For instance, MediaTek is rumored to be developing an exclusive AR chip for Meta.

System or device assembly is a key area of focus for Taiwan-based companies, particularly evident in the efforts of major ODMs like Quanta and Foxconn. These companies are enhancing their VR and AR hardware manufacturing through various strategies, including partnerships, mergers and acquisitions, and investment initiatives.

In the VR device supply chain, the strength of system assemblers lies in their ability to offer comprehensive product solutions, which expands the options available to prospective clients. The assembly of VR and AR devices presents unique challenges due to the necessity for high-quality image rendering and real-time motion capture. Numerous components are involved in the process.

Not all VR and AR device brands can develop head-mounted devices completely in-house, as demonstrated by companies like Meta and Sony. For newer market entrants, securing a comprehensive product solution that allows for future customization is a more desirable strategy. This demands that system assemblers have significant expertise in relevant technologies and ODM capabilities. As such, as opportunities in the VR and AR market continue to emerge, these assemblers are well-prepared to offer solutions for head-mounted devices.

Generative AI and Added-Value from Applications Will Sustain Future Growth Momentum

Beyond hardware, the focus on creating more content and valuable applications will be a major topic in the next phase of VR industry’s development, with generative AI poised to play a pivotal role.

Taking gaming as an example, VR game development is known to be exceedingly time-consuming, requiring developers to dedicate substantial amounts of time to coding. As a result, the games often lack diversity, customization, and meaningful game mechanics.

However, leveraging generative AI can expedite the game development process without sacrificing quality or increasing costs. Recent market analyses suggest that the adoption of generative AI could significantly reduce the time required to create XR learning modules from the 5-10 days typically seen in 2021 to less than 30 minutes today.

Consequently, major game engine providers like Unity are seizing this business opportunity. In mid-2023, Unity introduced a suite of generative AI development solutions tailored for VR game production. These solutions can be employed to create characters, objects, assets, and sound effects, thus significantly reducing development costs.

According to TrendForce’s research, global shipments of VR head-mounted devices are projected to register a slight year-on-year drop of 1.8%, but the annual total is still expected to surpass 9.3 million units.

Furthermore, with the releases of many new products ranging from chips and peripherals to complete systems, many of which were showcased at this year’s CES and MWC, there is strong bullish sentiment regarding the development of the VR industry. The strategies of major manufacturers for VR and AR devices also demonstrate intense efforts to explore new use cases beyond existing applications, or to expand into other commercial sectors such as remote assistance, virtual learning, and simulation training.

Additionally, in many countries, VR and AR are now being incorporated into medical treatments, such as psychological therapy and physical rehabilitation. Although the progress in promoting VR and AR technologies still depends on factors like pricing, specifications, and user experience, the expansion into new application markets is a positive development, particularly given the current shortage of content.

Therefore, the added-value provided by new applications will be a key determinant of the VR market’s growth momentum. Furthermore, the efficiency of using generative AI in content production holds the potential to propel device manufacturers into the next technological generation.

Join the AI grand event at Computex 2024, alongside CEOs from AMD, Intel, Qualcomm, and ARM. Discover more about this expo! https://bit.ly/44Gm0pK

(Photo credit: Apple)

On May 20, a report by Reuters revealed that Google plans to invest an additional Euro 1 billion in its data center park in Finland. This move aims to expand the scale and boost its AI business growth in Europe.

The report notes that in recent years, many data centers have been established in Nordic countries due to the cool climate, tax incentives, and ample supply of renewable energy. Finland’s wind power capacity has seen significant growth over these years, up by 75% to 5,677 megawatts by 2022, which brings electricity prices even down to negative values on particularly windy days.

Thus, Data center operators like Google have been taken advantage of this renewable energy, and already signed long-term wind power purchase agreements in Finland.

Driven by the AI wave, cloud providers such as Microsoft, Google, Meta, and Amazon have an increasingly robust demand for AI servers and data centers.

According to a previous forecast by TrendForce, considering the global CSPs’ demands for high-end AI servers (Those equipped with NVIDIA, AMD, or other high-end ASIC chips included) in 2024, the demands from four major U.S. CSPs: Microsoft, Google, AWS, and Meta are expected to account for 20.2%, 16.6%, 16%, and 10.8% of global demand respectively, reigning over the global market with a total proportion of more than 60%.

Read more

• [Insights] Big Four CSPs Continue to Shine in Q1 2024 Financial Reports, AI Returns Garnering Attention

(Photo credit: Google)

At the Google I/O 2024 developer conference on Tuesday, Google unveiled its 6th generation custom chip, the Trillium TPU, which is scheduled to hit the market later this year, according to the report by TechCrunch.

According to the information provided by Google on its website, compared to TPU v5e, Trillium boasts a 4.7x peak compute performance increase per chip. Google has also doubled the High Bandwidth Memory (HBM) capacity and bandwidth, along with a 1x increase in Interchip Interconnect (ICI) bandwidth between chips.

Additionally, Trillium features the third-generation SparseCore, a dedicated accelerator for processing large embeddings, aimed at handling advanced ranking and recommendation workloads. Moreover, Trillium achieves a 67% higher energy efficiency compared to TPU v5e.

Trillium has the capacity to expand up to 256 TPUs within a singular pod boasting high bandwidth and low latency. Additionally, it incorporates multislice technology, allowing Google to interlink thousands of chips, thus constructing a supercomputer capable of facilitating a data center network capable of processing petabits of data per second.

In addition to Google, major cloud players such as AWS, Meta, and Microsoft have also made their way to develop their own AI Chips.

In late 2023, Microsoft unveiled two custom-designed chips, the Microsoft Azure Maia AI Accelerator, optimized for AI tasks and generative AI, and the Microsoft Azure Cobalt CPU, an Arm-based processor tailored to run general purpose compute workloads on the Microsoft Cloud. The former is reportedly to be manufactured using TSMC’s 5nm process.

In May 2023, Meta also unveiled the Meta Training and Inference Accelerator (MTIA) v1, its first-generation AI inference accelerator designed in-house with Meta’s AI workloads in mind.

AWS has also jumped into the AI chip market. In November, 2023, AWS released Trainium2, a chip for training AI models.

Read more

• [News] Tech Giants Pick up Steam in AI Sector
• [News] Rise of In-House Chips: 5 Tech Giants In the Front

(Photo credit: Google)