AI investment costs and funding concerns... ‘GPU and AI infrastructure optimization’ draws attention as a solution

[Financial News]  As debate grows over overheating in AI investment and concerns mount about financing large-scale infrastructure buildouts, improved GPU utilization and AI model optimization technologies are drawing attention as ways to cut costs. Before bringing in new equipment, companies are looking to improve the efficiency of infrastructure investments by reducing idle capacity in existing GPUs and selecting AI models suited to each task.
On the 15th, major foreign media outlets reported that Hock Tan, CEO of Broadcom, said on the 12th that capital would be the biggest constraint facing companies as the AI industry develops. Tan also criticized speculative investment in AI, saying some investors had excessive expectations without sufficiently understanding what they were investing in.
As the burden of investing in AI infrastructure grows, overseas companies are focusing on GPU resource management and model optimization technologies. NVIDIA Corporation’s GPU resource management platform, ‘Run,’ allocates GPUs according to workloads and priorities and allows multiple tasks to share resources. This approach reduces idle GPU capacity and wait times, increasing the throughput and investment efficiency of existing infrastructure.
Investment is also flowing into AI model optimization companies. According to Business Insider, OpenRouter announced at the end of May that it had raised $113 million. Concentrate AI also secured at least $5 million in funding. OpenRouter helps users select suitable AI models based on factors including task type, model performance and cost.
In South Korea, companies with technologies for improving AI infrastructure efficiency are also targeting the market.
Acryl supplies GPUBASE, a GPU operations and optimization platform linked to its AI platform, Jonathan. Its features include GPU resource partitioning, workload scheduling, communication optimization and integrated monitoring. The company said it had conducted large-scale performance testing in Google Cloud (GCP), Microsoft Azure and Amazon Web Services (AWS) environments, and had also supplied GPUBASE to modular data centers in South Korea, among others. Nota Inc. uses its AI model optimization platform, NetsPresso, to help streamline models for specific hardware environments. Quantization and pruning reduce model size and computational demands, while allowing users to compare performance and quality before and after optimization on actual devices.
Lablup Inc. supplies Backend.AI, an AI infrastructure operations platform. Using GPU virtualization technology, it enables multiple users and tasks to share a single physical GPU. Its core function is to increase the utilization of existing equipment and reduce the need to buy additional GPUs.
The semiconductor industry is focusing on improving memory power efficiency. SK hynix said it had increased the power efficiency of High Bandwidth Memory 4 (HBM4) by at least 40% compared with the previous generation. The company says this can ease data-processing bottlenecks and reduce data center power costs.
Samsung Electronics Co., Ltd. also said it had improved HBM4 energy efficiency by about 40% compared with the previous generation by applying low-voltage design and power distribution optimization technologies. It expects this to support GPU computing performance while reducing power and cooling costs for servers and data centers.
An industry insider added, "As investment in AI infrastructure continues to expand, GPU utilization and power efficiency are emerging as key factors determining the economics of investment. Beyond simply expanding computing capacity, how much work can be handled using existing infrastructure and how much operating costs can be reduced are emerging as factors that determine the competitiveness of AI companies." 


[email protected] Kim Gyeong-a Reporter