OpenAI’s ChapGPT, Microsoft’s Copilot, Google’s Bard, and latest Elon Musk’s TruthGPT – what will be the next buzzword for AI? In just under six months, the AI competition has heated up, stirring up ripples in the once-calm AI server market, as AI-generated content (AIGC) models take center stage.

The convenience unprecedentedly brought by AIGC has attracted a massive number of users, with OpenAI’s mainstream model, GPT-3, receiving up to 25 million daily visits, often resulting in server overload and disconnection issues.

Given the evolution of these models has led to an increase in training parameters and data volume, making computational power even more scarce, OpenAI has reluctantly adopted measures such as paid access and traffic restriction to stabilize the server load.

High-end Cloud Computing is gaining momentum

According to Trendforce, AI servers currently have a merely 1% penetration rate in global data centers, which is far from sufficient to cope with the surge in data demand from the usage side. Therefore, besides optimizing software to reduce computational load, increasing the number of high-end AI servers in hardware will be another crucial solution.

Take GPT-3 for instance. The model requires at least 4,750 AI servers with 8 GPUs for each, and every similarly large language model like ChatGPT will need 3,125 to 5,000 units. Considering ChapGPT and Microsoft’s other applications as a whole, the need for AI servers is estimated to reach some 25,000 units in order to meet the basic computing power.

As the emerging applications of AIGC and its vast commercial potential have both revealed the technical roadmap moving forward, it also shed light on the bottlenecks in the supply chain.

The down-to-earth problem: cost

Compared to general-purpose servers that use CPUs as their main computational power, AI servers heavily rely on GPUs, and DGX A100 and H100, with computational performance up to 5 PetaFLOPS, serve as primary AI server computing power. Given that GPU costs account for over 70% of server costs, the increase in the adoption of high-end GPUs has made the architecture more expansive.

Moreover, a significant amount of data transmission occurs during the operation, which drives up the demand for DDR5 and High Bandwidth Memory (HBM). The high power consumption generated during operation also promotes the upgrade of components such as PCBs and cooling systems, which further raises the overall cost.

Not to mention the technical hurdles posed by the complex design architecture – for example, a new approach for heterogeneous computing architecture is urgently required to enhance the overall computing efficiency.

The high cost and complexity of AI servers has inevitably limited their development to only large manufacturers. Two leading companies, HPE and Dell, have taken different strategies to enter the market:

• HPE has continuously strengthened its cooperation with Google and plans to convert all products to service form by 2022. It also acquired startup Pachyderm in January 2023 to launch cloud-based supercomputing services, making it easier to train and develop large models.
• In March 2023, Dell launched its latest PowerEdge series servers, which offers options equipped with NVIDIA H100 or A100 Tensor Core GPUs and NVIDIA AI Enterprise. They use the 4th generation Intel Xeon Scalable processor and introduce Dell software Smart Flow, catering to different demands such as data centers, large public clouds, AI, and edge computing.

With the booming market for AIGC applications, we seem to be one step closer to a future metaverse centered around fully virtualized content. However, it remains unclear whether the hardware infrastructure can keep up with the surge in demand. This persistent challenge will continue to test the capabilities of cloud server manufacturers to balance cost and performance.

(Photo credit: Google)

AR/VR device shipments revised up to 14.19 million units in 2022, with an annual growth rate of 43.9%, according to TrendForce research. Growth momentum will come from increased demand for remote interactivity stemming from the pandemic, as well as Oculus Quest 2’s price reduction strategy. Microsoft HoloLens 2 and Oculus Quest 2 are first in market share for AR and VR, respectively.

According to TrendForce, the topic of the Metaverse has driven brands to actively plan for and stimulate product shipment performance. However, the AR/VR device market has yet to experienced explosive growth due to two factors: component shortages and the difficulty of developing new technologies. In addition, cosmetic and size considerations have made the more optically and technically difficult Pancake design the first choice for new high-end products. Furthermore, various embedded tracking feedback technologies key to enhancing the user’s immersive experience such as eye tracking and 6DoF further affect the development progress of a new product as a whole. Since there are no new foreboding products on the horizon, TrendForce believes, no other branded products have a chance at supplanting the current mainstream status of Oculus or Microsoft until at least 2023.

The Oculus Quest 2, which costs between US$200 and US$400, is currently the most popular AR/VR device in the consumer market. TrendForce expects Oculus to launch an advanced version of the Quest product within two years, reaching a hardware performance equivalent of US$700 or down to a retail price of US$500 with discounts. This product is expected to expand the size of the high-end consumer AR/VR market. The commercial market is dominated by the HoloLens 2 which costs more than US$1,000 and upwards of US$3,500. Since the commercial market places more emphasis on the benefits of hardware and software integration, manufacturers that dominate commercial systems, software, and platforms have the advantage. Thus, Apple has become another focus in the AR/VR device market.

Strong shipments of Oculus and Microsoft products will likely force Apple to release relevant products to join the competition this year. However, TrendForce states, considering hardware performance requirements and gross profit margins, Apple will likely target the commercial market and adopt the same pricing strategy as HoloLens, hardware priced in the thousands of dollars and a monthly subscription-based software solution. Overall, TrendForce believes that the launch of new products this year by Apple, Meta, and Sony may be delayed and will not add significant growth to the overall AR/VR market for the time being.

Factors such as the rising popularity of topics related to the metaverse and UGC (user-generated content), as well as the rapid increase in AR/VR device shipment, will likely result in the creation of a growing body of virtual reality content in the market, according to TrendForce’s latest investigations. TrendForce expects annual global virtual reality content revenue to grow at a 40% CAGR from US$2.16 billion in 2021 to US$8.31 billion in 2025.

TrendForce further indicates that gaming/entertainment, videos, and social interactions comprise the primary categories of virtual reality content. Incidentally, as the construction of the virtual world and the development of virtual reality content are unlikely to be accomplished by only a handful of companies alone, companies in this space will therefore place an increasing emphasis on UGC instead. Leading companies will likely leverage the build-out of virtual reality platforms/environments and the provisioning of developmental tools/interfaces in order to not only lower the barrier to entry for content creation, but also raise user participation, thereby driving up the content market for virtual reality applications.

In consideration of profitability, most companies still adopt a wait-and-see approach towards the virtual reality market because content development for the virtual world entails substantial time and expenses. The vast majority of UGC, however, is not profit-driven. Hence, TrendForce believes that UGC is likely a more suitable point of entry into the virtual reality market for most companies that wish to do so. Furthermore, companies that specialize in metaverse applications will place increasing emphasis on developing platforms, building comprehensive ecosystems, and lowering the barrier to entry for content creation through the appropriate development tools and interfaces.

On the whole, factors that affect the development of the global virtual reality content market include not only the availability of platforms and their respective contents, but also the build-out of hardware equipment and infrastructures, such as high-speed computing chip adoption as well as 5G and Wi-Fi 6 deployment. On the other hand, as the virtual world places a high demand on instant, lifelike, and stable interactions, the ability to resolve signal disruptions has in turn become a topic that demands attention. With regards to end devices, the penetration rate of AR/VR devices going forward will primarily be determined by suppliers’ pricing strategies. In light of the growth of virtual reality application content, companies will look to expand their user base via low-priced hardware devices and compensate for their reduced hardware profitability through software sales. Finally, in response to the demand for more immersive and interactive user experiences, the integration of more sensors and better feedback design is set to become the next major trend of AR/VR device development.

In light of the metaverse’s ability to satisfy the demands of WFH, virtual reality, and simulations, the smart manufacturing industry will also likely capitalize on the rise of the metaverse and undergo an accelerated growth of related technologies, according to TrendForce’s latest investigations. Global smart manufacturing revenue is expected to increase at a 15.35% CAGR across the 2021-2025 period and surpass US$540 billion in 2025. This growth can primarily be attributed to several factors. First, industrial applications take place in closed environments, and companies that utilize such applications have generally made good progress in terms of digital transformation. Furthermore, by utilizing simulation technologies, companies are able to significantly cut down on their labor costs, project time, and wasted resources. Simulation technologies, if developed as an industry 4.0 application, also serve as the backbone of CPS (cyber-physical systems). TrendForce therefore expects the smart manufacturing industry to be perfectly positioned with innate advantages and motivations as one of the main enablers of the metaverse.

Regarding the diverse mainstream smart manufacturing tools, digital twins, which major adopters believe to be a significant application of industry 4.0, empower the simulation of the physical world through digital data, bridge the virtual world with the real world, and subsequently serve as a key technology shaping the metaverse during its infancy. In particular, Microsoft has included digital twins in its metaverse technology stack due to their ability to generate rich digital models. It should be pointed out that the vast majority of digital twins currently used for industrial applications deliver digital simulations for either a single product or a single production line primarily because the reliability of simulated models requires a database containing sufficient data from the modeled product itself. Some examples of digital twins in action include Boeing utilizing digital twins to build engines, Unilever using simulated production lines to cut down on waste production, and Siemens Energy and Ericsson respectively leveraging Nvidia’s Omniverse platform to operate power plants and perform predictive maintenance as well as simulating equipment allocations for 5G networks.

Digital twin technologies will progress towards wider deployments and deeper operations in response to the rise of the metaverse and to the growing complexity of digital simulation models used for constructing products. Hence, relevant digital twin technologies will also begin to emerge in the market. In terms of width of deployment, digital twins need to model more comprehensive and extensive virtual objects and spaces that form the operating environment in the metaverse in order to achieve better predictive accuracy. Relevant technologies include 5G, WiFi 6, cloud and edge computing, smart sensors, as well as more resilient communication environments/computing platforms, and more diverse sensors. In terms of depth of operation, developments in technologies used for industrial drones, cobots, and machine vision feature improved precision and operability that enable AI-based decisions made in the virtual space to be applicable to decision-making scenarios in the real, physical world.

On the whole, taking into account the rapid development of AR/VR and HMI technologies, as well as other factors including economic outcomes, feasibility of operation, and the overall industrial environment, TrendForce believes that the direction of metaverse-based digital twin application development for industrial purposes will focus on human resource training, remote diagnostics, energy monitoring, and predictive maintenance in the short and medium terms. For instance, Rockwell, Siemens, ABB, Advantech, Ennoconn, and Delta are some of the companies that have made good progress in this area. In the long term, on the other hand, individual companies will likely be able to construct virtual factories in the collaborative industrial metaverse and thereby connect their various factory locations or even engage in cross-industrial collaborations. With regards to long-term applications, then, companies that are competent in industry 4.0 development and possess various lighthouse factories and vast databases will likely to be pioneers in the industry; leading examples include Bosch, Schneider Electric, Haier, and Foxconn.

Nvidia hosted its fall GTC (GPU Technology Conference) in early November, during which the company shared details regarding the progress that it had made on products and services such as AI software, data centers, automotive applications, and healthcare. In particular, Nvidia’s foray into virtual worlds and digital twins, both of which are closely tied to the metaverse, garnered significant attention from the public. By leveraging diverse simulation tools that reflect real-life circumstances, Nvidia has extended the application of virtual worlds from the local scale to the planetary scale, thereby reflecting the metaverse’s pioneering qualities and developmental progress.

Along with the ongoing metaverse craze, Nvidia also released its Omniverse Avatar technology platform as well as its Omniverse Replicator, which is a “synthetic data-generation engine” according to the company. Both of these releases are based on the Nvidia Omniverse, a platform that specializes in virtual collaboration. Whereas the Omniverse Avatar platform enables the creation of interactive virtual characters through synergies among voice AI technology, machine vision, and NLP (natural language processing), the Omniverse Replicator constructs more realistic, lifelike virtual worlds by training DNN (deep neural networks) using such synthetic data as velocity, depth, and weather conditions.

Digital twin-based virtual factories are starting to show the first hints of the metaverse

The metaverse value chain primarily revolves around commonly seen infrastructural backbones formed by telecommunications and cloud/edge computing. The virtual space that is then built on top of this infrastructure comprises HMI (human machine interface), decentralization, application creation, and user experiences. More specifically, HMI produces an AI-empowered immersive experience by combining multiple interactive technologies with an AR/VR base layer. At the moment, companies such as Nvida, Meta (formerly known as Facebook), Microsoft (including Xbox), and Vive are heavily invested in HMI development. Application creation, on the other hand, refers to mechanisms that make the metaverse more lively, reliable, diverse, and attractive. Some examples include graphical tools and cryptocurrency technologies. Representative groups focusing on this field include Roblox, IBM, Google AI, Epic, and Unity.

Regarding the content of Nvidia’s presentation during GTC apart from the Omniverse Avatar and Replicator, the company also released CloudXR, Showroom, and other Omniverse-based tools used for optimizing immersive experiences. As well, Nvidia also released the Modulus neural network model, which is accelerates the build-out of digital twins. These releases, in turn, demonstrates Nvidia’s competency and leadership in creating AI-driven software tools for the metaverse value chain. With regards to real-life use cases, digital twins currently represent most of Nvidia’s applications. For instance, BMW and Nvidia have partnered to construct a digital twin-based factory via the Omniverse platform capable of connecting ERP (enterprise resource management), shipment volume simulation, remote-controlled robots, production line simulation, etc. This partnership is indicative of promising early-stage growth of the metaverse.

Nvidia is extending its simulation application from factories to planets

While smart city development has remained one of the main use cases of simulation in recent years, Nvidia has further extended its simulation applications from use cases previously limited to singular offices or factory facilities. For instance, BIM (building information modeling) specialist Bentley Systems has teamed up with Nvidia to apply digital twins to public property management and maintenance. Ericsson, on the other hand, is utilizing Nvidia’s technology to construct a digital replica of an entire city for the purpose of checking 5G signal coverages, optimizing base station placement, and improving antenna designs. During the GTC, Nvidia unveiled the Earth-2 system, which is a supercomputer that generates a digital twin of planet earth for weather forecasts.

As a matter of fact, most products and services announced by Nvidia during GTC represent either a partial or entry-level application of the metaverse. However, as the post-pandemic new normal continues to drive up the demand for contactless and digital transformation applications, strengthening CPS (cyber physical systems) will remain one of the most significant trends in the market. As real-world environments become increasingly complex due to interactions among an increasing number of tools and use cases, Nvidia will aim to create a comprehensive framework for metaverse development through products/services based on more intelligent, comprehensive, and instant virtual worlds. Hence, TrendForce believes that Nvidia will need to address certain major challenges going forward, including lowering its tools’ usage barriers, strengthening its ecosystem, and attracting new users.

（Image credit: NVIDIA）