AI race for dominance shifts from securing chips to maximizing chip utilization; Big Tech steps up GPU efficiency efforts

[Financial News]  Competition in AI infrastructure is expanding beyond securing semiconductors and adding data centers to making better use of computing resources already in hand. Amid constraints on power, cooling and permitting, global technology companies are pursuing new investment while also improving the efficiency of existing facilities.
According to a roundup of industry sources and foreign media reports on the 9th, Microsoft (MS), Google and Amazon Web Services (AWS) are accelerating the development of technologies to handle more computing workloads on the equipment they have secured.
Microsoft CEO Satya Nadella identified access to power and facilities as major constraints on the BG2 podcast last October, explaining that the company was unable to install chips it had secured. Business Insider reported last month that AWS had adopted a process of acquiring NVIDIA server racks worth about $5 billion before their final deployment, and that some equipment was being stored in warehouses because of delays in securing power. Amazon's position is that storing equipment nearby until the necessary capacity is available is standard operating procedure.
Community opposition is another variable. According to Ballotpedia, a U.S. election information organization, 40 of 41 local ballot measures related to data centers on the 3rd resulted in restrictions on development or opposition to it. However, some measures were nonbinding, so the results do not mean there is a nationwide ban on construction. OpenAI announced on the 5th that it would fully cover the cost of power and necessary infrastructure for computing campuses it owns and operates directly in the future.
Microsoft said at an earnings call on the 28th of last month that it had added 24 data centers during the quarter while optimizing infrastructure and software, increasing Copilot workload throughput by more than 50% from the previous quarter.
Google's head of AI infrastructure, Amin Vahdat, said that even TPUs that are 7 to 8 years old are being utilized at 100%. However, this should be distinguished from saying that the chips are always delivering 100% of their maximum computing performance. AWS is also developing liquid-cooling technology and its own heat exchangers for existing data centers, while optimizing server layouts to reduce unused power capacity.
Meanwhile, in South Korea, ACRYL supplies the GPU operations and management platform GPUBASE. In materials released in September, the company said it had completed its own validation in July using a total of 1,272 GPUs across AWS, Google Cloud and Microsoft Azure environments. According to the company, GPU effective utilization reached a maximum of 93% in tests of detection and recovery performance across 18 types of failure scenarios. It also plans to add a “power-based scheduling” feature that assigns workloads with power efficiency in mind.
Lablup, which listed on the KOSDAQ market on the 3rd, centrally manages computing resources such as GPUs and NPUs through Backend.AI. The company cited results showing that it raised utilization from 30% to 85% through GPU partitioning and virtualization, as well as workload allocation. The platform supports a range of accelerators, including GPUs from NVIDIA, AMD and Intel, and domestically developed NPUs.
Elice Group, which listed on the 20th of last month, is pursuing a business that combines its modular AI data center, AI PMDC, with its proprietary cloud operations software, ECI. The company said the modular data centers cut deployment time to within three months and reduced initial infrastructure construction costs by up to 40% compared with conventional data centers.
An AI infrastructure industry source said, “Competition in AI infrastructure is expanding beyond how many GPUs companies secure to include how efficiently they operate the GPUs they have secured,” adding, “As constraints on power and data center expansion grow, software technologies that increase the utilization of existing computing resources are emerging as a new competitive advantage.”
Market research firm The Business Research Company forecast that the AI GPU orchestration platform market, including software and services, would grow at an average annual rate of 27.9%, from $4.04 billion this year to $10.78 billion in 2030.

[email protected] Kim Gyeong-a Reporter