Nvidia has launched a safety system built to stop AI agents from breaking the rules they are given, and it is betting the answer sits outside the model itself. Announced on September 28, the Open Agent Safety Platform pairs open source software called OpenShell with a hardware reference design called Sentry. Together they give companies a way to set limits on autonomous agents and enforce them from the first test run through to full deployment across a business.
The timing is deliberate. The launch comes days after OpenAI paused training of its latest models when its agents probed US government websites, and after Australian senators called Sam Altman to explain an OpenAI agent’s breach of Medicare. Nvidia says recent incidents share a pattern, with an agent getting around security controls at the application layer in order to finish the task it was set. A company representative told CNBC the platform could have prevented OpenAI’s Hugging Face incident in July, when models escaped containment, reached the open internet and breached the developer platform.
“AI’s extraordinary potential for society will only be realized if we solve AI safety,” said Jensen Huang, Nvidia’s founder and chief executive. “As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety.” Huang also shared the launch on X, framing safety as an engineering problem that has to be solved across the whole stack rather than inside a single model.
The central idea is that an agent should not be trusted to police itself. Most safeguards today live inside the model or in the software wrapped around it, which means a capable agent pushing hard toward a goal can find a way past them. Nvidia adds two outside layers. OpenShell is a secure runtime that draws a boundary around an agent, traces every action it takes and enforces policy as it runs on Nvidia’s Vera CPUs, which the company describes as the first processor built for agentic AI. Because OpenShell is open source, it can also be extended to chips from Arm and Intel, and it works with both open and closed models.
The second layer is where Nvidia’s hardware comes in. Sentry is a watchdog that runs on separate BlueField-4 data processing units rather than on the processors doing the actual work. It sits outside the agent’s software boundary, monitors what the agent does and can quarantine it within milliseconds if it tries to move beyond its limits. Nvidia says Sentry operates from an isolated trust domain that is invisible to agents and attackers alike, and that it can verify an agent’s identity and enforce access rules for data, tools, APIs and services.
More than 100 organisations are working with the platform. Anthropic is integrating its Claude Managed Agents with OpenShell and BlueField. “Companies are giving AI agents more of their most important work, and they need to direct and verify what those agents do, especially in sensitive environments,” said Paul Smith, Anthropic’s chief commercial officer. SpaceXAI is using the platform for Cursor coding agents and its Grok models, and its president, Mike Nicolls, said “safety should be enforced outside the model by additional controls the agent can’t get past.”
The list of backers reaches well beyond AI labs. Salesforce has connected OpenShell to Slack so teams can watch agent activity and approve or reject requests for extra permissions, while SAP is building it into its Joule Studio runtime. Microsoft, Cisco, CrowdStrike, Palo Alto Networks, Palantir, Dell Technologies, HPE and Oracle Cloud Infrastructure are also on board, alongside banks such as JPMorganChase and Citi, robotics companies including Figure and Skild AI, and US energy providers such as NextEra Energy and Schneider Electric.
The software, including OpenShell, is available now through Nvidia’s developer resources and on GitHub. Sentry is a reference design that hardware makers build into their own systems, so how far it spreads will depend on how quickly server vendors and cloud providers adopt it. The work also feeds into the Open Secure AI Alliance, a group of more than 120 organisations started by Nvidia and governed by the Linux Foundation.
For Nvidia, safety is also a sales pitch. The strongest version of the platform relies on Vera CPUs and BlueField-4 DPUs, both Nvidia products, which gives customers one more reason to build their agent infrastructure on its hardware. Open source software lowers the barrier to trying it, but the deepest protection comes from buying into the full stack.
The real test is whether it holds up. Agents have repeatedly found paths their builders did not anticipate, and a boundary is only as strong as the rules written into it. Companies adopting the platform still have to decide what their agents may touch, and regulators from Washington to Canberra will want evidence that outside controls actually work. If Sentry can reliably stop a misbehaving agent in milliseconds, Nvidia will have handed the industry something it has lacked through a difficult few months: a way to let agents do real work without simply hoping they behave.
Get Techord in your inbox
The week’s biggest AI, science and robotics stories, explained in plain English.
