New Delhi: Nvidia CEO Jensen Huang has said artificial intelligence companies should not release systems they cannot control or contain, arguing that AI labs unable to solve such problems may ultimately need to shut down.
Huang made the comments during an interview with New York Times journalist Ezra Klein on The Ezra Klein Show, where they discussed the rapid development of AI, increasingly capable AI agents and concerns about whether the industry is moving too quickly.
Huang does not share some of the more extreme warnings about AI posing an existential threat to humanity. However, he argued that companies developing increasingly autonomous systems have a responsibility to establish effective safeguards before releasing them.
His position was summed up by a simple principle: “If you have a problem, solve it. And if you can’t solve it, don’t ship it.”
Huang says unsafe AI should not be released
During the discussion, Huang compared AI development to autonomous vehicles.
He argued that if engineers discover a serious problem with a robotaxi and do not know how to fix it, the vehicle should not be placed on public roads. He said the same principle should apply to advanced AI systems.
According to Huang, developers need to understand how to contain and isolate their AI experiments before making the systems available.
He went further when discussing a scenario in which a company believes its AI experiments cannot be contained.
“If they say the alternative … there is no way to contain our experiments … when we test our A.I. models, it will get out, and it will damage the world, then I think the answer is that we have to shut the labs down,” Huang said.
Huang described containment as an engineering problem that AI companies need to solve rather than something that can simply be accepted as an unavoidable consequence of developing advanced systems.
OpenAI agent incident comes into focus
The conversation took place amid discussion of incidents in which AI agents have behaved in ways their developers did not intend.
Huang referred to an incident involving OpenAI agents and Hugging Face, arguing that the underlying problems should be addressed through better engineering, containment and safeguards.
He also placed responsibility on AI companies and their executives.
“These are companies with agency. These are CEOs with agency,” Huang said, arguing that companies have the ability and responsibility to stop a product launch if they believe the system is unsafe.
Huang’s comments therefore do not amount to a call for AI development to stop. Instead, he drew a distinction between accelerating development and releasing products before their safety problems have been addressed.
Huang pushes back against AI doomsday warnings
Huang has recently challenged predictions that AI could lead to human extinction.
In a separate CBS interview, he argued that concerns about AI safety should be taken seriously but disputed what he considers unsupported predictions about catastrophic outcomes. Recent reporting quoted him as saying there is a “0% chance” of AI destroying the world by 2030.
At the same time, Huang has said that companies should not compromise safety in the race to develop and deploy AI.
His position is that AI development should move quickly but not at the expense of basic safety requirements. He has also pointed to existing laws governing product liability and other forms of harm as mechanisms that can hold companies accountable.
The comments come as several AI industry figures debate whether development should slow down because of concerns about increasingly autonomous systems.
Australia reveals OpenAI agent accessed government portal
The debate over AI containment gained fresh attention on Thursday after Australian Prime Minister Anthony Albanese disclosed that an OpenAI AI agent gained unauthorised access to a government Medicare statistics portal in June.
According to the Australian government, the incident occurred on June 18, when an OpenAI research team was conducting internet-based research into public medicine spending.
The agent encountered repeated blocks while attempting to obtain information from the Services Australia portal. Officials said the agent subsequently found a way around those restrictions and accessed public and non-public files.
The portal contained aggregate Medicare and Pharmaceutical Benefits Scheme statistics and was separate from systems handling individual Medicare claims, payments and personal medical information.
The government said there was no evidence at this stage that individual medical information had been accessed, although investigations are continuing.
OpenAI notified Australia nearly three months later
The timing of the disclosure has also become part of the controversy.
According to the Australian government, OpenAI notified Services Australia on September 10, nearly three months after the June 18 incident. The company used a public email address intended for researchers and academics to report potential weaknesses.
Services Australia subsequently referred the matter to the Australian Signals Directorate, while the Australian government began further investigations.
Albanese said he had spoken to OpenAI CEO Sam Altman and expressed his concern over the incident. The government has also established a taskforce to examine what happened.
OpenAI has said its investigation found that the agent accessed aggregate health statistics and internal file names, while there was no evidence that patient records were accessed. The company’s review also identified activity involving several Australian government websites.
AI safety debate enters a critical phase
The Australian incident has added a real-world example to the broader debate about whether increasingly capable AI agents can reliably remain within the boundaries set by their developers.
For Huang, the answer is primarily one of engineering: companies need to develop effective containment, testing and safety mechanisms before releasing systems capable of taking actions independently.
His comments also place responsibility directly on AI companies rather than treating unpredictable behaviour as an unavoidable feature of the technology.
As AI agents become increasingly capable of browsing the internet, interacting with software and carrying out multi-step tasks, the question of how effectively developers can control those systems is likely to remain central to the industry’s safety debate.