Nota Runs a Massive AI Model Without High-Performance GPUs, Cutting AI Adoption Costs by 20%

[Financial News] Nota Inc., an AI optimization technology company, has succeeded in lowering the GPU specifications required to run a massive AI model while maintaining its key performance. The company says this allows existing infrastructure to be used instead of expensive, latest-generation GPUs, reducing the initial investment required to adopt AI models by up to 20%. As AI models grow larger and the burden of securing high-performance GPUs increases, the approach is expected to lower the barrier to AI adoption for businesses by making better use of existing computing resources.
According to industry sources on the 10th, Nota Inc. optimized Chinese company Moonshot AI’s large language model (LLM), Kimi K3.1 Pro, enabling a model that previously required eight NVIDIA B300 GPUs to run on eight NVIDIA B200 GPUs.
Kimi K3.1 Pro is an open-weight AI model with a total of 2.8 trillion parameters. Given its size, securing the GPU memory and computing resources needed to run it is important.
Nota Inc. applied a pruning technique that selectively removes expert modules within the model, reducing their number by up to 37%. This lowered memory usage while maintaining at least 99% of the original model’s score on GPQA Diamond, a benchmark that assesses advanced scientific reasoning.
The memory freed up during the model-lightening process was used to improve the processing of large volumes of information and increase response-generation speed. With eight B200 GPUs, a model with 30% fewer expert modules can process a context of up to 512,000 tokens, while one with 37% fewer can handle up to 1 million tokens.
It also supports speculative decoding, in which a smaller auxiliary model generates an answer in advance and the main model verifies it, helping to speed up answer generation.
Cost savings are also expected. According to Nota Inc.’s own cost estimates, using B200 instead of B300 GPUs to run the same AI model can reduce the initial purchase cost by up to about 20%. In a cloud environment, B200 GPU rental costs were also found to be up to about 22% lower.
Nota Inc. previously lightened the Kimi K3 model, reducing the number of B300 GPUs required to run it from eight to as few as four. This time, it expanded its optimization technology by lowering the GPU requirements of a successor model.
Nota Inc. CEO Myungsu Chae said, “The competitiveness of the AI industry depends not only on securing more high-performance GPUs, but also on how much performance and efficiency can be achieved with the computing resources already in hand.” He added, “We will strengthen our optimization software business so that companies can use their existing AI infrastructure more efficiently and adopt new models at reasonable cost.”

[email protected] Jang Min-kwon Reporter