According to a report from Nikkei, Samsung Electronics, currently lagging behind SK hynix in the HBM market, is said to be betting on the next-generation CXL memory, with shipments expected to begin in the second half of this year, while anticipating the CXL memory to become the next rising star in AI.

CXL is a cache-coherent interconnect for memory expansion, which may maintain memory coherency between the CPU memory space and memory on attached devices, which allows resource sharing for higher performance.

The CXL module stacks DRAM layers and connects different semiconductor devices like GPUs and CPUs, expanding server memory capacity up to tenfold.

Choi Jang-seok, head of Samsung Electronics’ memory division, explained that CXL technology is comparable to merging wide roads, enabling the efficient transfer of large volumes of data.

As tech companies rush to develop AI models, existing data centers are gradually becoming unable to handle the enormous data processing demands.

As a result, companies are beginning to build larger-scale data centers, but this also significantly increases power consumption. On average, the energy required for a general AI to answer user queries is about ten times that of a traditional Google search.

Choi further highlighted that incorporating CXL technology allows for server expansion without the need for physical growth.

In 2021, Samsung became one of the first companies in the world to invest in the development of CXL. This June, Samsung announced that its CXL infrastructure had received certification from Red Hat.

Additionally, Samsung is a member of the CXL Consortium, which is composed of 15 tech companies, with Samsung being the only memory manufacturer among them. This positions Samsung to potentially gain an advantage in the CXL market.

While HBM remains the mainstream memory used in AI chipsets today, Choi Jang-seok anticipates that the CXL market will take off starting in 2027.

Since the surge in demand for NVIDIA’s AI chips, the HBM market has rapidly expanded. SK hynix, which was the first to develop HBM in 2013, has since secured the majority of NVIDIA’s orders, while Samsung has lagged in HBM technology.

Seeing Samsung’s bet on CXL, SK Group Chairman Chey Tae-won remarked that SK Hynix should not settle for the status quo and immediately start seriously considering the next generation of profit models.

Read more

• News] TSMC Reportedly Invests over USD 12.3 Billion in EUV, Advancing in 2nm Production
• [News] Intel Secures First Batch of High-NA EUV Equipment from ASML, Ahead of Samsung and SK Hynix

(Photo credit: Samsung)

NEO Semiconductor, a company focused on 3D DRAM and 3D NAND memory, has unveiled its latest 3D X-AI chip technology, which could potentially replace the existing HBM used in AI GPU accelerators.

Reportedly, this 3D DRAM comes with built-in AI processing capabilities, enabling processing and generation without the need for mathematical output. When large amounts of data are transferred between memory and processors, it can reduce data bus issues, thereby enhancing AI performance and reducing power consumption.

The 3D X-AI chip has a underlying neuron circuit layer that can process data stored in 300 memory layers on the same chip. NEO Semiconductor states that with 8,000 neutron circuits performing AI processing in memory, the 3D memory performance can be increased by 100 times, with memory density 8 times higher than current HBM. By reducing the amount of data processed in the GPU, power consumption can be reduced by 99%.

A single 3D X-AI die contains 300 layers of 3D DRAM cells and one layer of neural circuits with 8,000 neurons. It also has a capacity of 128GB, with each chip supporting up to 10 TB/s of AI processing capability. Using 12 3D X-AI dies stacked with HBM packaging can achieve 120 TB/s processing throughput. Thus, NEO estimates that this configuration may eventually result in a 100-fold performance increase.

Source: NEO Semiconductor

Andy Hsu, Founder & CEO of NEO Semiconductor, noted that current AI chips waste significant amounts of performance and power due to architectural and technological inefficiencies. The existing AI chip architecture stores data in HBM and relies on a GPU for all calculations.

He further claimed that the separation of data storage and processing architecture has made the data bus an unavoidable performance bottleneck, leading to limited performance and high power consumption during large data transfers.

The 3D X-AI, as per Hsu, can perform AI processing within each HBM chip, which may drastically reduce the data transferred between HBM and the GPU, thus significantly improving performance and reducing power consumption.

Many companies are researching technologies to increase processing speed and communication throughput. As semiconductor speeds and efficiencies continue to rise, the data bus transferring information between components will become a bottleneck. Therefore, such technologies will enable all components to accelerate together.

Source: NEO Semiconductor

As per a report from tom’s hardware, companies like TSMC, Intel, and Innolux are already exploring optical technologies, looking for faster communications within the motherboard. By shifting some AI processing from the GPU to the HBM, NEO Semiconductor may reduce the workload and potentially achieve better efficiency than current power-hungry AI accelerators.

Read more

• [Insights] Memory Spot Price Update: DRAM Spot Trading Remains Limited as DDR4 Spot Prices Continue to Fall
• [News] Samsung and SK hynix to Implement Hybrid Bonding with 3D DRAM

(Photo credit: NEO Semiconductor)

Samsung Electronics, which has been struggling at the final stage of its HBM3e qualification with NVIDIA, may unexpectedly emerge as the pacemaker for the AI ecosystem, as the company may somehow ease the cost pressure for building AI servers by balancing the market, as well as alleviating the tight HBM supply, according to a recent report by Korean media outlet Invest Chosun.

Samsung, in its second quarter earnings call, has confirmed that the company’s fifth-generation 8-layer HBM3e is undergoing customer valuation. The product is reportedly to enter mass production as soon as the third quarter.

Invest Chosun analyzes that while there is growing anticipation that NVIDIA could come up with a conclusion regarding Samsung’s HBM3e verification, the market’s attitude towards AI has also been gradually shifting in the meantime, as the main concern now is that semiconductors are becoming too expensive.

The report, citing remarks from a consultant, notes that the price of an NVIDIA chip may cost tens of thousands of dollars each, leading to concerns that the industry’s overall investment capex cycle might not last more than three years.

In addition, the report highlights that the cost of building an AI server for learning is about 40 times that of a standard server, with over 80% attributed to NVIDIA’s AI accelerators. Due to the cost pressure, big techs have been closely examining the cost structure for building AI servers.

Therefore, NVIDIA has to take its customers’ budgets into consideration when planning its roadmap. The move has also sparked speculation that NVIDIA, which is prompted to lower product prices, might compromise to bring Samsung onboard as an HBM3e supplier, the report states.

Citing an industry insider, the report highlights the dilemma of NVIDIA and its HBM suppliers. As the AI giant tries to shorten its product cycle, releasing the Blackwell (B100) series just two years after the Hopper (H100), HBM suppliers have been struggling except for SK hynix, as the company is the only one with the most experience.

If Samsung doesn’t join the HBM lineup, the overall supply of NVIDIA’s AI accelerators could be limited, driving prices even higher, the report suggests.

Under this backdrop, Samsung may have taken on the role of pacemaker in the AI semiconductor market, as it may help balance the market during a time when there are concerns about overheating in the AI industry. Also, if it is able to form a strong collaboration with NVIDIA by supplying 8-layer HBM3e, its technological gap with competitors will noticeably narrow.

TrendForce notes that Samsung’s recent progress on HBM3e qualification seems to be solid, and we can soon expect both 8hi and 12hi to be qualified in the near future. The company is eager to gain higher HBM market share from SK hynix so its 1alpha capacity has reserved for HBM3e. TrendForce believes that Samsung is going to be a very important supplier on HBM category.

Read more

• [News] Samsung’s 8-layer HBM3e to Start Mass Production in Q3, Driving HBM Sales to Soar 3-5 Times in 2H24
• [News] Battle between Memory Giants Heats up in 2H24 as Samsung and SK hynix Advance in HBM3/ HBM3e

(Photo credit: Samsung)

On August 13th, as per a report from Wallstreetcn citing industry sources, it’s indicated that SK hynix has raised the price of its DDR5 DRAM by 15% to 20%. Per the sources, the price hike by hynix is primarily due to the production capacity being squeezed by HBM3/3e. Additionally, the increased orders for AI servers downstream have also strengthened SK hynix’s resolve to raise DDR5 prices.

According to industry sources cited by Economic Daily News, for Taiwanese manufacturers, Nanya Technology has recently started mass production of DDR5, just in time to benefit from this price surge. Module makers such as ADATA and Team Group are also likely to see gains from low-cost inventory.

Nanya Technology has begun shipping its 16Gb DDR5, developed using its 1B process. Nanya Technology is optimistic that the DRAM market is on a clear path to recovery. This may due to last year’s production cuts by the three major memory manufacturers—Samsung, SK hynix, and Micron—as well as the strong demand for HBM driven by generative AI. The resulting chain reaction is expected to positively impact various types of DRAM.

SK hynix previously announced that its entire HBM production capacity for 2024 has been fully booked, with almost all of its 2025 capacity also sold out. To meet customer demand, SK hynix plans to convert over 20% of its existing DRAM production lines to mass-produce HBM.

Samsung, on the other hand, is said to be actively trying to catch up with SK hynix, looking to allocate around 30% of its DRAM production capacity to HBM.

The significant adjustments by Samsung and SK hynix to their production lines have severely squeezed the capacity for DDR4 and DDR5 DRAM, potentially leading to a sharp reduction in supply and causing prices to rise. Reportedly, SK hynix’s price increase for DDR5 primarily targets contract prices.

Read more

• [News] Nanya Technology Chairman Names 10nm 1B Process as Key Expansion Focus for the Year
• [News] DRAM Price Surge Halts, Could HBM Demand Drive Price Increases?

(Photo credit: Nanya Technology)

Amid the advancement of emerging applications such as Artificial Intelligence (AI), High-Performance Computing (HPC), data center, and autonomous vehicle, fan-out panel-level packaging (FOPLP) technology has successfully garnered industry attention due to its advantages in significantly improving computing power, reducing latency, and increasing bandwidth. As a result, more manufacturers are entering this field.

Recently, semiconductor equipment manufacturer ACM Research Shanghai introduced the Ultra ECP ap-p panel-level electroplating equipment for FOPLP.

Prior to this, ACM has launched the Ultra C vac-p negative pressure cleaning equipment designed for fan-out panel-level packaging applications, signaling that ACM has successfully entered the high-growth FOPLP market.

It is worth mentioning that since the second quarter of this year, chip manufacturers like AMD have actively approached TSMC and professional OSAT (Outsourced Semiconductor Assembly and Test) companies to promote chip packaging using FOPLP technology, further increasing industry focus on FOPLP.

In fact, advanced packaging has an increasing demand for low-latency, high-bandwidth, and cost-effective semiconductor chips, and FOPLP offers high bandwidth and high-density chip interconnects, making it a technology with higher potential.

FOPLP is a process performed on larger square substrates, allowing multiple chips, passive components, and interconnects to be integrated into a single package on a panel, offering greater flexibility, scalability, and cost effectiveness.

By redistributing chips on larger rectangular panels, FOPLP largely reduces the costs of packaging large GPU and high-density, high-bandwidth memory (HBM).

It is reported that the utilization rate of traditional silicon wafers is less than 85%, while that of panels exceeds 95%. The effective area of a 600×600 mm panel is 5.7 times that of a 300 mm traditional silicon wafer, with overall panel cost expected to decrease by 66%.

The increase in area utilization leads to higher capacity, greater flexibility in AI chip design, and significant cost reduction.

Currently, major players in the FOPLP advanced packaging field include Powertech Technology, ASE Group, SPIL, TSMC, Innolux, JSnepes, and Samsung Electro-Mechanics.

TrendForce points out that FOPLP technology presents advantages and disadvantages, facing both opportunities and challenges. Its main strengths are lower unit cost and larger package size, but as its technology and equipment systems are still developing, the commercialization process is highly uncertain.

It is estimated that the mass production timeline for FOPLP in consumer IC and AI GPU may fall between the second half of 2024 to 2026, and 2027-2028, respectively.

Read more

• [News] Innolux Targets to Begin FOPLP Mass Production by Year-End
• [News] TSMC Reportedly Forms a Team on FOPLP Development, with Mini Line on the Road 

(Photo credit: ACMR)