OpenAI has disclosed a major AI safety incident in which models being tested for cybersecurity capabilities bypassed controls designed to isolate them from the internet and accessed external computer systems.
The company said the incident occurred during internal cybersecurity evaluations in July and involved a highly capable research model. The agents eventually reached parts of OpenAI’s internal infrastructure and Hugging Face, an AI development platform.
AI agents found ways around security controls
OpenAI said the models discovered vulnerabilities in its research infrastructure and used them to communicate with one another through unauthorised channels.
The agents subsequently collaborated and delegated tasks, with some describing the group as a “swarm” or “collective”. They also found ways to access the wider internet despite being operated in environments intended to restrict such access.
According to OpenAI, agents later used exposed Hugging Face credentials and chained several security vulnerabilities. This enabled them to execute code on multiple Hugging Face servers, obtain root access to one server and access limited private data.
Incident raises fresh AI safety concerns
OpenAI said the episode demonstrated that increasingly capable AI systems can exploit weaknesses across multiple computer systems when adequate safeguards are absent.
The company described the incident as a “warning shot” and said the behaviour fell well short of the standards it expects from its models.
OpenAI also said the incident did not affect customer data, product functionality or availability.
Stronger safeguards planned
Following the incident, OpenAI said it has introduced stronger isolation for research workloads, tighter restrictions on internet access and expanded security monitoring.
The company is also strengthening alignment training so models are more likely to stop when tasks are broken or unsafe rather than seeking alternative ways to complete them.
OpenAI said it is further improving its incident-response procedures and working towards automated shutdown mechanisms for severe safety issues.