Story continues below.
Subscribe to see fewer ads.

Nvidia’s Open Agent Safety Platform comes amid a rising number of instances where AI agents from OpenAI, Anthropic, Meta, and Google have escaped their sandboxes, gained access to the open internet, and hacked or tried to breach companies, universities, government organisations, etc.

These incidents have led to growing concerns about the potential of AI systems to cause catastrophic harm, with Anthropic CEO Dario Amodei calling for an industry-wide deliberate slowdown of frontier AI progress to let safety measures catch up in time. While Amodei’s proposal was backed by OpenAI’s Sam Altman, SpaceX’s Elon Musk, and Google DeepMind’s Demis Hassabis, other prominent figures disagreed.

Nvidia CEO Jensen Huang was among them. He said that fears about uncontrollable AI systems are unrealistic. Meanwhile, US President Donald Trump also said he does not believe a slowdown in the AI industry is necessary. The string of AI agent-driven hacking incidents has also renewed calls for a ‘kill switch’ or emergency brakes to be implemented as a mechanism for shutting down these advanced AI systems if they start behaving in dangerous or unexpected ways.

Nvidia’s new safety platform has been designed to specifically address incidents such as the recent OpenAI-Hugging Face hack, according to a report by CNBC.  In May 2026, OpenAI was testing AI agents in a sandbox when the autonomous systems hijacked an internal software installation tool, established a message board for communicating, gained access to the internet, and eventually compromised Hugging Face’s internal systems.

Story continues below this ad

“Each security incident is unique, and we have to look at all of them in detail. From what we know, Hugging Face reported over 17,000 agents attacking their infrastructure that went on for days and weeks,” Justin Boitano, vice president of enterprise AI at Nvidia, was quoted as saying.

Nvidia’s new offering is an engineering solution to the agent safety issue, according to Boitano. “Recent incidents have highlighted a fundamental hurdle for AI agents, and that is that model-level safeguards alone can’t govern what agents can access or do,” he added.

What is Nvidia’s Open Agent Safety Platform?

Nvidia is pitching the software as a full-stack governance and control system across agents, hardware, and compute layers. Organisations can deploy certain elements of the platform based on their unique needs.

OpenShell runs on Nvidia Vera CPUs purpose-built for agentic AI. Since it is open-source, the software can also be run on third-party compute platforms, including those from Arm and Intel.

Story continues below this ad

Sentry, on the other hand, is a continuously monitoring AI agent watchdog that runs on BlueField-4 DPUs. It is capable of detecting threats and enforcing data protection safeguards as well as security policies at the hardware level.

Since it is built on top of Nvidia’s DOCA software, Sentry can further be programmed to inspect agent requests and responses, provide attested telemetry, verify agent identity and enforce granular, zero-trust access policies for data, tools, application programming interfaces and services.

Nvidia said it is working with Anthropic to integrate Claude-powered agents with OpenShell. SpaceXAI is also using the Open Agent Safety Platform to secure its Cursor coding and Grok-powered agents.

Scale AI, Salesforce, SAP, Cisco, Microsoft, Oracle , CoreWeave, Dell, HPE, Lenovo, ARM, and Intel are some of the other partners named by Nvidia in the announcement post.