OpenAI’s ChapGPT, Microsoft’s Copilot, Google’s Bard, and latest Elon Musk’s TruthGPT – what will be the next buzzword for AI? In just under six months, the AI competition has heated up, stirring up ripples in the once-calm AI server market, as AI-generated content (AIGC) models take center stage.

The convenience unprecedentedly brought by AIGC has attracted a massive number of users, with OpenAI’s mainstream model, GPT-3, receiving up to 25 million daily visits, often resulting in server overload and disconnection issues.

Given the evolution of these models has led to an increase in training parameters and data volume, making computational power even more scarce, OpenAI has reluctantly adopted measures such as paid access and traffic restriction to stabilize the server load.

High-end Cloud Computing is gaining momentum

According to Trendforce, AI servers currently have a merely 1% penetration rate in global data centers, which is far from sufficient to cope with the surge in data demand from the usage side. Therefore, besides optimizing software to reduce computational load, increasing the number of high-end AI servers in hardware will be another crucial solution.

Take GPT-3 for instance. The model requires at least 4,750 AI servers with 8 GPUs for each, and every similarly large language model like ChatGPT will need 3,125 to 5,000 units. Considering ChapGPT and Microsoft’s other applications as a whole, the need for AI servers is estimated to reach some 25,000 units in order to meet the basic computing power.

As the emerging applications of AIGC and its vast commercial potential have both revealed the technical roadmap moving forward, it also shed light on the bottlenecks in the supply chain.

The down-to-earth problem: cost

Compared to general-purpose servers that use CPUs as their main computational power, AI servers heavily rely on GPUs, and DGX A100 and H100, with computational performance up to 5 PetaFLOPS, serve as primary AI server computing power. Given that GPU costs account for over 70% of server costs, the increase in the adoption of high-end GPUs has made the architecture more expansive.

Moreover, a significant amount of data transmission occurs during the operation, which drives up the demand for DDR5 and High Bandwidth Memory (HBM). The high power consumption generated during operation also promotes the upgrade of components such as PCBs and cooling systems, which further raises the overall cost.

Not to mention the technical hurdles posed by the complex design architecture – for example, a new approach for heterogeneous computing architecture is urgently required to enhance the overall computing efficiency.

The high cost and complexity of AI servers has inevitably limited their development to only large manufacturers. Two leading companies, HPE and Dell, have taken different strategies to enter the market:

• HPE has continuously strengthened its cooperation with Google and plans to convert all products to service form by 2022. It also acquired startup Pachyderm in January 2023 to launch cloud-based supercomputing services, making it easier to train and develop large models.
• In March 2023, Dell launched its latest PowerEdge series servers, which offers options equipped with NVIDIA H100 or A100 Tensor Core GPUs and NVIDIA AI Enterprise. They use the 4th generation Intel Xeon Scalable processor and introduce Dell software Smart Flow, catering to different demands such as data centers, large public clouds, AI, and edge computing.

With the booming market for AIGC applications, we seem to be one step closer to a future metaverse centered around fully virtualized content. However, it remains unclear whether the hardware infrastructure can keep up with the surge in demand. This persistent challenge will continue to test the capabilities of cloud server manufacturers to balance cost and performance.

(Photo credit: Google)

Microsoft are among the major technology companies that are now undergoing a major round of layoffs. Having released around 11,000 employees, Microsoft has downsized staffing across its numerous business operations, including gaming units Xbox and Bethesda. This round of layoffs has also seen the disbanding of the development teams behind Altspace VR and the Mix Reality Tool Kit. The VR social media platform Altspace VR is scheduled to shut down on March 10th this year.

Is Microsoft Exiting the Market for Technologies Related to AR/VR and Metaverse?

Microsoft acquired AltspaceVR in 2017, and the Windows Mix Reality headset was released in the same year. The plan was to have the newly acquired social media platform generate the demand for the headset, thereby spurring other hardware brands to continue investing in similar products.

However, fast forward to the current year of 2023, the Windows Mix Reality headset possesses almost no market share as AltspaceVR fails to hold a notable number of active users for this device. TrendForce believes Microsoft has no choice but to stop maintaining the social media platform that is not bringing in any significant economic benefits.

However, TrendForce has to point out that “platform system” is still the core of Microsoft’s strategy for the development of AR/VR technologies. Going forward, Microsoft intends to have applications related to VR social media integrated into the Microsoft Mesh, which is its new VR/AR communication and collaboration platform that can work on multiple types of devices.

Microsoft can also encourage third-party developers to build VR social media platforms that are compatible with its technologies. Bottom line is this: there is no need for Microsoft to operate its own VR social media platform. All in all, AltspaceVR is a component that Microsoft has taken out because it no longer fits into its strategy. Terminating the platform does not mean that the company has decided to sit on the sideline in the formation of the Metaverse market, as speculated by some outsiders.

The Metaverse Is Only a Medium, and Maintaining User Engagement on a VR Social Media Platform Depends on the Capability to Offer a Variety of Functions

The shutdown of AltspaceVR reveals the current challenges in the development of Metaverse communities. In the case of social media services, simply adding AR/VR technologies or some elements related to the Metaverse will not lead to long-term engagement by users. To get users to stay, these platforms need to rely on their own special interfaces, functions, and features.

Take the relatively successful VR social media platforms such as Roblox, Rec Room and VRChat as examples. They first enable users to self-generate content and express their creativity in various ways, and then they provide the avenue for social interactions and trading of virtual goods. Hence, TrendForce believes fulfilling the creative aspiration is the key to keeping users. Offering some AR/VR technologies and gimmicks associated with the Metaverse is just not enough.

On the other hand, functions that allow creativity tend not to be the reason why the majority of users join a particular social media platform in the first place. Also, a platform that has to work with an AR/VR device will be relatively difficult to access and operate, and this further limits the size of its user base. Taking the aforementioned factors in account, it is clear as to why AltspaceVR failed to gain traction. Positioning itself only as a social media platform, it did not really stand out in the competition even with AR/VR functions.

TrendForce’s takeaway from the closure of Altspace is that rather than building a social media service from scratch, Microsoft’s interests would be best served by acquiring an existing social media app or platform that already has a huge following. With the support from the Microsoft Mesh, such app or platform would be able to strengthen Microsoft’s service offerings for Metaverse communities in the future.

（Image credit: Microsoft LinkedIn）

Sony stated in an earnings call that PS5 shipments totaled only 11.5 million units in the last fiscal year (2Q21 to 1Q22), missing the target of 14.8 million units. Increasing PS5 game console production will be the company’s main task for now and sales are expected to rise to 18 million units this fiscal year. In addition, Sony also stated that it will strengthen live broadcast services and extend game services to other devices, as well as step up its VR business.

PS5 sales will continue to grow in 2022 but market instability remains

Although Sony had high expectations going into the launch of the PS5 and market reception was good, the PS5’s final sales volume was stifled by production hamstrung as a result of component shortages. Therefore, Sony specifically mentioned in the Business Segment Briefings, comparing US retailer events sold PS4 to the PS5, the PS4 sold an average of 6 units per minute, while the PS5 now sells approximately 1,000 units per minute, demonstrating the strong market demand for PS5.

At the beginning of the PS5 release, there were reports of an insufficient supply of components. When the PS5 was released at the end of 2020, it had been in production for several months and accumulated a certain amount of inventory. Even if production capacity was in short supply when the console was released, Sony could still meet some market demand in the first year with its inventory and then increase production capacity in 2021 to make up the difference. However, in 2021, semiconductor production capacity was also in short supply. Not only game console products, but numerous other products experienced a shortage of components. Naturally, Sony was unable to further increase the supply of PS5 consoles, resulting in PS5 sales coming in lower than originally expected. Sales even declined in 1Q22. As the imbalance between supply and demand of semiconductors gradually eases in 2022, Sony predicts that PS5 production will begin to increase to fulfill market demand and announced a sales target for this fiscal year of 18 million units.

Even so, there remain many uncertainties in 2022. First of all, although pandemic lockdowns in China have yet to directly affect the assembly and production of game consoles in Shandong, the risk of potential fallout still exists given the uncertainty of pandemic development. Secondly, demand from the overall consumer market may be exhausted in 2022. This is due to unfavorable factors such as inflation, wars, pandemics, and rising energy prices. Disposable income in 2022 is forecast to shrink as a result and force consumers to give up some non-essentials or delay purchases. So this may cause consumers to delay replacing their current console with a PS5. A combination of the original dearth of PS5 supply and the impact of the scalper market seriously depleted the stock of products on the market. Some consumers who were eager to enjoy new console games may have switched over to buying an Xbox Series X/S first which may contribute to PS5’s 2022 annual sales volume falling short of Sony’s estimate. TrendForce expects that volume will only reach 14.34 million units. Even if this forecast references the range of Sony’s fiscal year (2Q22 to 1Q23), estimated sales volume will only increase to 16 million units.

（Image credit: Pixabay）

According to TrendForce research, corporate demand for digital transformation including artificial intelligence and high-performance computing has accelerated in recent years, which has led to increasing adoption of cloud computing. In order to improve service flexibility, the world’s major cloud service providers have gradually introduced ARM-based servers. The penetration rate of ARM architecture in data center servers is expected to reach 22% by 2025.

In the past few years, ARM architecture processors have matured in the fields of mobile terminals and Internet of Things but progress in the server field has been relatively slow. However, companies have diversified cloud workloads in recent years and the market has begun to pay attention to the benefits ARM architecture processing can provide to data centers. TrendForce believes that ARM-based processors have three major advantages. First, they can support diverse and rapidly changing workloads and are more scalability and cost-effective. Second, ARM-based processors provide higher customization for different niche markets with a more flexible ecosystem. Third, physical footprint is relatively small which meets the needs of today’s micro data centers.

Influenced by geopolitics and the strengthening of data sovereignty in various countries, major cloud service providers and telecom operators are actively developing micro data centers which will further drive the penetration of ARM-based processors. At the same time, from the perspective of cloud service providers currently adopting ARM-based processors, Graviton, led by AWS, has the largest market scale and began encroaching gradually into the market in 2021. TrendForce also observed that AWS’s deployment of ARM-based processors in 2021 reached 15% of overall server deployment and will exceed 20% in 2022. This forces other major cloud service providers to keep up by initiating their own projects at various foundries. If testing is successful, these projects are expected to start mass introduction in 2025.

In addition, according to the Neoverse Platform plan previously released by ARM, its Platform Roadmap will also be one of the key drivers of penetration. This product line is set up to target ultra-large-scale data centers and edge computing infrastructure. However, it is worth mentioning, since x86 is still mainstream in the market and ARM-based server CPU suppliers only maintain small-batch production orders at this stage and primarily focus on ultra-large-scale data centers, introduction of ARM-based servers into enterprise data centers will be slow going. Thus, TrendForce believes that it will still be difficult for ARM-based servers to compete with x86-based servers before 2025.

AR/VR device shipments revised up to 14.19 million units in 2022, with an annual growth rate of 43.9%, according to TrendForce research. Growth momentum will come from increased demand for remote interactivity stemming from the pandemic, as well as Oculus Quest 2’s price reduction strategy. Microsoft HoloLens 2 and Oculus Quest 2 are first in market share for AR and VR, respectively.

According to TrendForce, the topic of the Metaverse has driven brands to actively plan for and stimulate product shipment performance. However, the AR/VR device market has yet to experienced explosive growth due to two factors: component shortages and the difficulty of developing new technologies. In addition, cosmetic and size considerations have made the more optically and technically difficult Pancake design the first choice for new high-end products. Furthermore, various embedded tracking feedback technologies key to enhancing the user’s immersive experience such as eye tracking and 6DoF further affect the development progress of a new product as a whole. Since there are no new foreboding products on the horizon, TrendForce believes, no other branded products have a chance at supplanting the current mainstream status of Oculus or Microsoft until at least 2023.

The Oculus Quest 2, which costs between US$200 and US$400, is currently the most popular AR/VR device in the consumer market. TrendForce expects Oculus to launch an advanced version of the Quest product within two years, reaching a hardware performance equivalent of US$700 or down to a retail price of US$500 with discounts. This product is expected to expand the size of the high-end consumer AR/VR market. The commercial market is dominated by the HoloLens 2 which costs more than US$1,000 and upwards of US$3,500. Since the commercial market places more emphasis on the benefits of hardware and software integration, manufacturers that dominate commercial systems, software, and platforms have the advantage. Thus, Apple has become another focus in the AR/VR device market.

Strong shipments of Oculus and Microsoft products will likely force Apple to release relevant products to join the competition this year. However, TrendForce states, considering hardware performance requirements and gross profit margins, Apple will likely target the commercial market and adopt the same pricing strategy as HoloLens, hardware priced in the thousands of dollars and a monthly subscription-based software solution. Overall, TrendForce believes that the launch of new products this year by Apple, Meta, and Sony may be delayed and will not add significant growth to the overall AR/VR market for the time being.