Samsung Electronics has unveiled a blueprint to evolve AI-era high-bandwidth memory (HBM) into an “A..

Hotchips 2026 presents HBM evolution stage 3 roadmap
“BaseDye as SoC” memory conversion declaration
GPU memory controllerHBM, responsible for some of the operations

사진 확대 Han Sang-wook, Technical Leader (TL), a DRAM design team of Samsung Electronics’ memory division, is presenting at the semiconductor conference ‘Hot Chips 2026’ held at Stanford University on the 23rd (local time). [Photo = correspondent Won Hoseob]

Samsung Electronics has unveiled a blueprint to evolve AI-era high-bandwidth memory (HBM) into an “AI system semiconductor” rather than a memory that stores simple data. It also proposed a new structure in which HBM takes charge of some AI operations as well as memory control functions of GPUs, and in the long run, GPUs and memory are directly stacked as one.

Han Sang-wook, a technical leader (TL) of Samsung Electronics’ memory division DRAM design team, said, “The study began when asked why HBM base dies are not used like System-on-Chip (SoC) if they are made into advanced logic processes,” adding, “If existing HBMs were only in charge of data delivery, they would develop into customized HBMs that perform much more diverse functions in the future.”

One TL diagnosed that the bottleneck of AI semiconductors, in particular, is no longer just in computational power but in memory structure. “The semiconductor industry is facing physical limitations of single chip size and interposer size limitations,” he said. “The solution is to actively utilize HBM base dies already placed around GPUs.”

The first method proposed is to move the memory controller inside the GPU to the HBM. A TL said, “Memory controllers account for about 5-10% of the SoC area,” adding, “If it is transferred to HBM, the secured space can be filled with operation cores, and performance can be expected to improve by 10-20%.” He then said, “Most of Samsung’s customers are actively pushing for moving memory controllers to custom HBMs.”

Samsung went one step further and proposed a structure in which HBM performs some AI operations. One TL said, “It is difficult to put heavy operations in HBM base dies because of heat generation,” but explained, “If operations with a lot of memory access are selectively transferred to HBM, data movement can be greatly reduced, which can dramatically increase system power efficiency.” He introduced this structure as “aHBM (Advanced HBM).”

He also emphasized that the larger the AI model, the greater the memory capacity. The next-generation AI architecture requires not only bandwidth but also memory capacity at the same time, he said. “The structure that directly connects LPDDR or additional HBMs in the HBM base die can provide higher bandwidth and lower latency than the existing PCIe method.”

As a final step, Samsung also unveiled its “zHBM” vision of stacking GPUs and HBMs vertically. “The AI industry is moving quickly toward combining memory and computing more closely,” one TL said. “The ultimate goal is to get rid of the 2.5D interposer and connect GPUs and DRAMs directly vertically.” “If this structure is implemented, DRAM power can be reduced by about 70% compared to HBM5,” he said. Samsung explains that the bandwidth will also increase 2.3 times compared to the previous generation (HBM4E). [Silicon Valley correspondent Wonho-seop]