“AI that searches for information but comes up with off-base answers” ... Korea University (KU) finds a “blind spot” in AI agents that think for themselves

[Financial News] As technology that enables artificial intelligence (AI) to find information it needs online, make judgments, and provide answers on its own becomes more widespread, Korean researchers have uncovered a security vulnerability that allows just a few externally manipulated documents to alter AI’s decision-making path and lead it to the wrong conclusion.
KU said on the 28th that a team led by Jang Buru, a professor in its Department of Computer Science, had identified a new security attack technique, “KidnapRAG,” which disrupts an AI agent’s search and reasoning process step by step to steer it toward an answer intended by an attacker.
Agentic RAG (agent-based retrieval-augmented generation), a technology that goes beyond simply answering questions and enables AI to search the internet for information on its own and reach an answer through multiple steps of reasoning, is being widely adopted by major AI services such as ChatGPT and Gemini. However, concerns have been raised that the overall decision-making process could be distorted if AI reads maliciously crafted “bait documents” while autonomously searching for information.
Professor Jang’s team analyzed a method for gradually changing the direction of AI’s reasoning using only manipulated documents posted externally, without directly attacking or hacking into an AI service’s internal systems. The method takes advantage of the fact that AI reveals its search results and intermediate reasoning steps externally: it predicts in advance what information AI will search for next and places manipulated documents at each juncture.
The experiments showed that this type of attack persistently prevented AI from returning to the correct reasoning path it had initially established, achieving an attack success rate as high as 78%. Unlike previous attack techniques, which AI could detect and correct on its own, the newly identified technique effectively hijacks the AI’s decision-making chain itself, the research team explained.
The study is significant because it lays the groundwork for proactively assessing AI security safeguards as the use of AI agents expands into fields such as finance, medicine, and law, where accuracy is vital.
Jang Buru, a professor at KU, said, “We have timely demonstrated that AI agents’ search and reasoning paths can be attacked under conditions similar to those in real-world service environments.” He added, “I hope this will lead to defensive technologies that protect every stage of AI agents’ operation and contribute to the development of safer and more trustworthy AI.”
The study was conducted in collaboration with KT Frontier Labs and with support from the Institute for Information and Communications Technology Planning and Evaluation (IITP). It will be presented at EMNLP 2026, a prestigious international conference in natural language processing, being held in Budapest, Hungary, through the 29th. 

[email protected] Kim Man-gi Reporter