MS unveils 'Guardian,' an AI sheriff that monitors AI [Global AI Briefing]

[Financial News] Microsoft (MS) has unveiled a security framework to monitor the behavior of artificial intelligence (AI) agents working in businesses and control risky actions. As AI agents move beyond simply handling tasks on people’s behalf and gain direct access to corporate data and systems, MS believes AI itself must be managed as a new internal security threat. On the 19th (local time), MS announced on its official blog that it would add the Guardian Agents feature to Project Perception, its security system. Guardian Agents are designed to continuously monitor the behavior of other AI agents operating in businesses and respond when risks increase.

Until now, corporate AI security has focused mainly on situations in which external hackers use AI to launch attacks or break into corporate systems. But new problems have emerged as AI agents gain access to internal documents and workplace tools and take on real tasks, including approvals and analysis. Even AI that has been properly deployed can follow incorrect instructions or be influenced by malicious information from outside, leading it to take unintended actions. If an AI with excessive access privileges makes a mistake, the damage could spread within the company.
Microsoft executive vice president overseeing security Hayete Gallot stressed in an official announcement, “Advanced AI models are unpredictable regardless of their source, so their behavior needs to be continuously monitored and controlled.”
Guardian Agents are tasked with detecting and responding to risky actions before an AI takes them.
To that end, MS is working to establish a system that gives each AI agent a unique digital identity and grants it only the minimum privileges required for its work. It will also adjust access privileges as the level of risk changes, based on an analysis of the AI’s behavior and surrounding circumstances.
For example, if an AI handling expense approvals tries to violate the approval rules, its privileges could be restricted or human approval could be required. If the risk becomes serious, isolating the AI or shutting it down are also possible responses.
As AI agents have rapidly emerged as actors in hacking, calls are growing in the global AI market to separate AI models’ access privileges and have separate security systems control their behavior. With AI capabilities advancing rapidly, a key challenge for companies adopting AI is who will detect it when an AI makes a poor decision and how far its actions should be restricted.
[email protected] Lee Gu-soon Reporter