HBM’s Dominance to Continue...“CXL Is Merely a Supplement to Storage”

[Financial News] The dominance of high-bandwidth memory (HBM), a core technology and bottleneck for artificial intelligence (AI) services, is expected to continue for the foreseeable future.
Technical experts from Intel and OpenAI who took part in a panel discussion at the AI Infrastructure Summit, held on the 16th (local time) at the Santa Clara Convention Center in Santa Clara, California, expressed negative views about Compute Express Link (CXL), which has drawn attention as a potential alternative to HBM.
CXL is a next-generation interconnect standard designed to link AI accelerators, which handle computing tasks, with memory more efficiently.
Some in the semiconductor industry had expected that the full-scale adoption of CXL would replace a substantial portion of increasingly expensive HBM and significantly reduce the cost of building AI servers.
Daniel Morris, an OpenAI researcher responsible for designing AI accelerators, said unequivocally during the panel discussion, "From the standpoint of actually running AI models, I cannot find a use for CXL." He added, "If I had to identify a role, it would be to set aside inactive data that is not frequently used in large models," and acknowledged, "Contrary to expectations, I have not seen any use cases specialized for running models in practice."
Vidiya Thiagarajan, Intel’s head of system-on-chip (SoC) architecture, also assessed, "It is useful to pool memory through CXL, but this does not replace HBM. It merely supplements secondary storage."
He pointed out, "The speed of data moving to and from a graphics processing unit (GPU) through CXL is by no means as fast as HBM."
In other words, no matter how much the interconnect method improves, CXL is unlikely to displace HBM because of the difference in bandwidth.
Accordingly, Samsung Electronics and SK hynix, which effectively control the global HBM market, are expected to maintain their dominant positions for some time to come.

[email protected] Lee Seok-woo, International Affairs Correspondent Reporter