SK hynix: "Processing-in-memory technology will resolve AI data bottlenecks"

[Financial News] SK hynix presented a vision for resolving AI data bottlenecks by leveraging processing-in-memory (PIM) technology, which performs computations directly inside memory semiconductors.
Eui-Cheol Lim, executive vice president in charge of Solution AI at SK hynix, said during a presentation at the AI Infrastructure Summit held at the Santa Clara Convention Center in Santa Clara, California, on the 16th (local time) that PIM would overcome the limitations of AI infrastructure centered on graphics processing units (GPUs) and high-bandwidth memory (HBM).
Unlike conventional systems, which transfer data from memory to an AI accelerator such as a graphics processing unit (GPU) for computation and then send it back to memory, PIM allows processing units inside the memory to handle simple calculations directly.
Samsung Electronics also unveiled PIM technology targeting on-device AI at Hot Chips, a semiconductor conference held last month.
Lim noted that the AI industry currently relies mainly on SRAM when ultra-fast inference is required, saying, "SRAM-based systems provide high bandwidth, but they have clear limitations in terms of capacity and cost."
He added, "PIM provides approximately 300 times the capacity of SRAM in the same area while maintaining high internal bandwidth."
Regarding limitations previously cited for PIM, including insufficient memory capacity and higher unit costs resulting from larger circuit areas, Lim emphasized that these issues could be addressed through hybrid bonding technology, which separately manufactures memory and compute dies and vertically connects them.
He also introduced the concept of expanding PIM beyond individual chips to the server-rack level.
Lim stressed that the industry must move beyond an era of single-infrastructure systems that process all AI services using only a combination of GPUs and HBM. He called for hardware innovation across the entire memory hierarchy, including PIM and high-bandwidth flash (HBF), along with software optimization and integrated design.


[email protected] International Affairs Specialist Lee Seok-woo Reporter