On July 1st, according to a report from Reuters, the French antitrust authority plans to file charges against NVIDIA, accusing the company of engaging in anti-competitive practices, making France the first country to take such action against NVIDIA.

The French competition regulator had raided NVIDIA’s local offices in September last year. At the time, they did not disclose the details of the investigation or the company involved, only stating it was related to the graphics card sector.

However, as per a previous report from Bloomberg, NVIDIA claimed that the French agency collected information from them regarding their business and competition in the graphics card and cloud service provider market as part of an ongoing inquiry into competition in those markets.

Sources cited by Reuters’ report indicated that last year’s raid was part of a broader investigation into cloud computing. With the surge in global chip demand following the advent of ChatGPT, NVIDIA, as the world’s largest manufacturer of AI and computer graphics cards, has naturally attracted close scrutiny from antitrust authorities in Europe and the United States.

NVIDIA previously disclosed in regulatory filings that both EU and French regulators had requested information about its graphics card products. The French antitrust authority has been actively investigating to understand NVIDIA’s key role in AI processors, its pricing policies, chip shortages, and the impact on prices.

Last Friday, the French authorities released a report on competition in generative AI, highlighting the risk of chip suppliers abusing their power. The report pointed out concerns about the chip industry’s heavy reliance on NVIDIA’s CUDA software for chip programming. Additionally, NVIDIA’s focus on investing in AI cloud service provider CoreWeave has also raised significant concerns among the authorities.

Reportedly, it is understood that companies violating French antitrust rules could face fines of up to 10% of their global annual revenue, though they can choose to make concessions to avoid penalties.

Moreover, the European Commission is currently gathering informal feedback to determine if NVIDIA has breached its antitrust rules, although it has not yet launched a formal investigation into anti-competitive behavior.

On the other hand, the New York Times reported on June 5th that the U.S. Department of Justice and the Federal Trade Commission (FTC) have reached an agreement, led by senior officials of both agencies, over the past week. The DOJ will investigate whether NVIDIA has violated antitrust laws, while the FTC will examine the conducts of OpenAI and Microsoft.

Read more

• [News] NVIDIA’s H200 Order Delivered from Q3, Boosting Server Supply Chain with Strong Demand
• [News] US Government Reportedly Investigating Microsoft, NVIDIA, and OpenAI for AI Industry Monopolization

(Photo credit: NVIDIA)

The New York Times reported on June 5th that the U.S. Department of Justice (DOJ) and the Federal Trade Commission (FTC) have reached an agreement, led by senior officials of both agencies, over the past week. The DOJ will investigate whether NVIDIA has violated antitrust laws, while the FTC will examine the conducts of OpenAI and Microsoft.

Reportedly, Jonathan Kanter, who is said to be the top antitrust official in the DOJ’s Antitrust Division, highlighted at an AI conference at Stanford University last week that AI’s reliance on massive amounts of data and computing power gives dominant companies a significant advantage. In a February interview, FTC Chair Lina Khan stated that the FTC aims to identify potential issues in the early stages of AI development.

As per Reuters’ report, Microsoft, OpenAI, NVIDIA, DOJ and FTC did not immediately respond to requests for comment outside regular business hours.

In a May interview with CNBC, Appian co-founder and CEO Matt Calkins stated that AI might not be a winner take all market. He suggested that if alliances could secure victory in the AI race, Google would already have won.

Per a report from Roll Call on May 15th, a bipartisan Senate AI working group led by Senate Majority Leader Chuck Schumer released an AI roadmap, calling for the federal government to invest at least USD 32 billion annually in non-defense-related AI systems.

In March, The Information reported that Microsoft does not want its hiring of Inflection AI’s two co-founders and the majority of its 70-member team to be perceived as an acquisition.

Read more

• [News] A Recap of NVIDIA’s Four Major Application Areas of New Technology from Jensen Huang
• [News] Micron Slightly Raises Capital Expenditure for 2024, HBM Expected to Further Drive Revenue Growth in 2025

(Photo credit: NVIDIA)

Currently, the issue of low power consumption remains a key concern in the industry. According to a recent report by the International Energy Agency (IEA), given that an average Google search requires 0.3Wh and each request to OpenAI’s ChatGPT consumes 2.9Wh, the 9 billion searches conducted daily would require an additional 10 terawatt-hours (TWh) of electricity annually. Based on the projected sales of AI servers, AI industry might see exponential growth in 2026, with power consumption needs at least ten times that of last year.

Ahmad Bahai, CTO of Texas Instruments, per a previous report from Business Korea, stated that recently, in addition to the cloud, AI services have also shifted to mobile and PC devices, leading to a surge in power consumption, and hence, this will be a hot topic.

In response to market demands, the industry is actively developing semiconductors with lower power consumption. On memory products, the development of LPDDR and related products such as Low Power Compression Attached Memory Module (LPCAMM) is accelerating. These products are particularly suitable for achieving energy conservation in mobile devices with limited battery capacity. Additionally, the expansion of AI applications in server and automotive fields is driving the increased use of LPDDR to reduce power consumption.

In terms of major companies, Micron, Samsung Electronics, and SK Hynix are speeding up the development of the next generation of LPDDR. Recently, Micron announced the launch of Crucial LPCAMM2. Compared to existing modules, this product is 64% smaller and 58% more power-efficient. As a low-power dedicated packaging module that includes several latest LPDDR products (LPDDR5X), it is a type of LPCAMM. LPCAMM was first introduced by Samsung Electronics last year, and it is expected to enjoy significant market growth this year.

Currently, the Joint Electron Device Engineering Council (JEDEC) plans to complete the development of LPDDR6 specifications within this year. According to industry news cited by the Korean media BusinessKorea, LPDDR6 is expected to start commercialization next year. The industry predicts that LPDDR6’s bandwidth may more than double that of previous generation.

Read more

• [News] South Korea Reportedly Unveils USD 19 Billion Semiconductor Subsidy to Boost R&D and Infrastructure Investment
• [News] US Allocates USD 39 Billion Subsidy to Semiconductor Industry for Establishing Plants

(Photo credit: SK Hynix)

Could AI Be Heading Towards an “Energy Crisis”? Speculation suggests that a Microsoft engineer involved in the GPT-6 training cluster project has warned that deploying over 100,000 H100 GPUs in a single state might trigger a collapse of the power grid. Despite signs of OpenAI’s progress in training GPT-6, the availability of electricity could emerge as a critical bottleneck.

Kyle Corbitt, co-founder and CEO of AI startup OpenPipe, revealed in a post on social platform X that he recently spoke with a Microsoft engineer responsible for the GPT-6 training cluster project. The engineer complained that deploying InfiniBand-level links between GPUs across regions has been a painful task.

Continuing the conversation, Corbitt asked, “why not just colocate the cluster in one region?” The Microsoft engineer replied, “Oh yeah, we tried that first. We can’t put more than 100K H100s in a single state without bringing down the power grid.”

At the just-concluded CERAWeek 2024, attended by top executives from the global energy industry, discussions revolved around the advancement of AI technology in the sector and the significant demand for energy driven by AI.

As per a report from Bloomberg, during his speech, Toby Rice, chief of the largest US natural gas driller, EQT Corp., cited a forecast predicting AI could gobble up more power than households by 2030.

Additionally, Sam Altman from OpenAI has expressed concerns about the energy, particularly electricity, demands of AI. Per a report from Reuters, at the Davos Forum earlier this year, he stated that AI’s development requires breakthroughs in energy, as AI is expected to bring about significantly higher electricity demands than anticipated.

According to a report by The New Yorker on March 9th citing data of Alex de Vries, a data expert at the Dutch National Bank, it has indicated that ChatGPT consumes over 500,000 kilowatt-hours of electricity daily to process around 200 million user requests, equivalent to over 17,000 times the daily electricity consumption of an average American household. As for search giant Google, if it were to use AIGC for every user search, its annual electricity consumption would increase to around 29 billion kilowatt-hours, surpassing the annual electricity consumption of countries like Kenya and Guatemala.

Looking back at 2022, when AI hadn’t yet sparked such widespread enthusiasm, data centers in China and the United States respectively accounted for 3% and 4% of their respective total societal electricity consumption.

As global computing power gradually increases, a March 24th research report from Huatai Securities predicts that by 2030, the total electricity consumption of data centers in China and the United States will reach approximately 0.95/0.65 trillion kilowatt-hours and 1.7/1.2 trillion kilowatt-hours respectively, representing over 3.5 times and 6 times that of 2022. In an optimistic scenario, by 2030, the AI electricity consumption in China/US will account for 20%/31% of the total societal electricity consumption in 2022.

Read more

• [News] NVIDIA’s H100 AI Chip No Longer Out of Reach, Inventory Pressure Reportedly Forces Customers to Resell
• [Insights] OpenAI CEO’s Ambition to Build Semiconductor Plant Grows Stronger

(Photo credit: Taiwan Business Topics)

Microsoft is reportedly developing a customized network card for AI servers, as per sources cited by global media The Information. This card is expected to enhance the performance of its in-house AI chip Azure Maia 100 while reducing dependency on NVIDIA as the primary supplier of high-performance network cards.

Leading this product initiative at Microsoft is Pradeep Sindhu, co-founder of Juniper Networks. Microsoft acquired Sindhu’s data center technology startup, Fungible, last year. Sindhu has since joined Microsoft and is leading the team in developing this network card.

According to the Information, this network card is similar to NVIDIA’s ConnectX-7 interface card, which supports a maximum bandwidth of 400 Gb Ethernet and is sold alongside NVIDIA GPUs.

Developing high-speed networking equipment tailored specifically for AI workloads may take over a year. If successful, it could reduce the time required for OpenAI to train models on Microsoft AI servers and lower the costs associated with the training process.

In November last year, Microsoft unveiled the Azure Maia 100 for data centers, manufactured using TSMC’s 5-nanometer process. The Azure Maia 100, introduced at the conference, is an AI accelerator chip designed for tasks such as running OpenAI models, ChatGPT, Bing, GitHub Copilot, and other AI workloads.

Microsoft is also in the process of designing the next generation of the chip. Not only is Microsoft striving to reduce its reliance on NVIDIA, but other companies including OpenAI, Tesla, Google, Amazon, and Meta are also investing in developing their own AI accelerator chips. These companies are expected to compete with NVIDIA’s flagship H100 AI accelerator chips.

Read more

• [Insights] Microsoft Unveils In-House AI Chip, Poised for Competitive Edge with a Powerful Ecosystem
• [Insights] AI Market: A Battleground for Tech Giants as Six Major Companies Develop AI Chips

(Photo credit: Microsoft)