How Can AI Performance Be Improved Without Adding GPUs? ... OchestrO to Host Webinar

[Financial News] OchestrO will introduce ways to optimize inference infrastructure to reduce low GPU utilization and response delays that arise in operating artificial intelligence (AI) services. It will share technologies that improve AI service performance and operational efficiency by making more effective use of existing resources instead of acquiring additional GPUs.
AI infrastructure software specialist OchestrO announced on the 26th that it will host an online webinar at 2 p.m. on the 27th on the topic “Improving AI Service Performance: Should You Buy More GPUs First?”
As generative AI spreads across industries, the importance of inference and service operations, as well as model training, is growing. While companies are increasing investment in securing GPUs, low GPU utilization, response delays and operating costs remain challenges to be addressed.
In particular, as agentic AI becomes more widespread, there are more cases in which AI models are called multiple times to complete a single task. Using the same GPU resources for input processing and answer generation can create competition between tasks and delay responses. OchestrO plans to explain at the webinar how to reduce bottlenecks in inference and operate GPU resources efficiently.
The webinar will cover distributed inference using OchestrO’s AI inference operations platform, “Concerto AI (CONCERTO A.I.),” ways to optimize GPU resources, and performance validation results.
Concerto AI is a platform that reduces competition for GPU resources by separating the input processing (Prefill) and answer generation (Decode) stages of large language models (LLMs). It automates the construction and deployment of inference infrastructure and provides integrated performance monitoring.
According to OchestrO’s performance validation results, Concerto AI increased throughput by 62.8% in high-load environments while using the same GPU resources as conventional model-serving methods. Token generation latency fell by 52.1%.
The company will also present examples of infrastructure optimization integrating NetApp storage technology. It will showcase cases in which GPU memory demands were reduced and the efficiency of serving concurrent users was improved through KV cache offloading based on GPU Direct Storage (GPUDirect Storage, GDS). The company said model-weight loading time was cut from 71 seconds to 8 seconds, while the time to put new servers into service fell from 187 seconds to 80 seconds.
OchestrO plans to use Concerto AI to support the entire process from AI service development to operations, and to expand its AI development environment, model operations and management, and integrated monitoring capabilities. It also intends to support a range of hardware, including NVIDIA GPUs and Korean-developed neural processing units (NPUs), helping companies operate their AI infrastructure independently.
OchestrO is also pursuing partnerships in the sovereign AI infrastructure sector. Its affiliate OchestrO Cloud has signed memorandums of understanding with four companies, including Furiosa AI and Upstage Co., Ltd., and is jointly developing a Korean NPU-based “Sovereign AI Full-Stack Appliance.”
The webinar will be held online via AllShow TV starting at 2 p.m. on the 27th. Participation is free, and attendees can join after registering in advance.
“In the AI era, operational capabilities that draw the greatest possible performance from the resources we already have are becoming more important than how many GPUs we secure,” said Kim Beom-jae, CEO of OchestrO. “At this webinar, we will share Concerto AI’s technology and performance validation results and offer practical solutions to help companies get more out of their GPU investments and operate AI services reliably.”
[email protected] Yoon Hong-jip Reporter