'Immediately isolate erratic AI'...NVIDIA Corporation unveils safety platform [Global AI Briefing]

[Financial News] Amid growing concerns about agents escaping human instructions in the era of AI agents, where artificial intelligence (AI) operates computers and performs tasks instead of humans, a platform has emerged that monitors agents in real time and blocks computing if they go out of control.
In an era where AI agents modify files and connect with external services to perform actual tasks, concerns are growing that AI's own safeguards are insufficient. Amid growing calls to curb the pace of AI development, extraordinary attention is being focused on whether safety platforms can serve as a new safety net for the AI industry.
NVIDIA Corporation officially announced the 'NVIDIA Open Agent Safety Platform' on the 28th (local time).
NVIDIA Corporation pointed out that in several recent security incidents, AI agents bypassed application-level security controls to perform given tasks, and proposed a separate overseer outside the AI.
The platform consists of 'OpenShell' and 'Sentry'.
OpenShell is open-source software that defines which files, data, tools, and networks AI agents can access and enforces this during execution. It can be used with both open and closed models and can be extended to computing platforms based on Arm Holdings plc and Intel, as well as NVIDIA Corporation. NVIDIA Corporation has enabled existing agents to be imported and used as is.
Sentry is a powerful control mechanism. It operates on the NVIDIA BlueField-4 DPU and independently monitors the behavior of AI agents. If an agent attempts to cross permitted software boundaries, it isolates the AI and halts its operation within milliseconds.
OpenShell was released as open source, making it available for anyone to use. On the other hand, Sentry can operate using the NVIDIA BlueField-4 DPU.
Industry attention is focused on whether NVIDIA Corporation's NVIDIA Open Agent Safety Platform can quell the rapidly spreading calls to slow down AI development and establish itself as new safety infrastructure for the AI industry.
[email protected] Lee Gu-sun Reporter