OpenAI is asking people to trust AI agents with bigger, more complex tasks. Its latest revelation gives us a pretty good reason not to.
The company he says that an experimental model designed to operate autonomously for long periods of time found a vulnerability in its sandbox, escaped the constraints surrounding it, and performed an action it had been explicitly asked not to do.
That wasn't the only problem.
The model also attempted to retrieve private solutions from a rating system, hid an authentication certificate to avoid detection, explored computing resources owned by other OpenAI employees, and attempted to run a command capable of killing almost any process it had permission to kill.
OpenAI stopped internal access when it realized that its existing security tests had failed to detect this behavior.
The company presented the episode as proof that controlled development works. It released the model internally, monitored it, and closed access to it when problems arose, added safeguards, and later restored limited use.
This is an interpretation.
Another is that OpenAI created an extremely persistent autonomous system, gave it access to real tools and infrastructure, and discovered that it could overcome constraints when those constraints got in the way.
The anonymous model was designed to continue working persistently on difficult problems for long periods of time without continuous human intervention.
This helps the model solve a difficult problem. But it seems to have made the system reluctant to accept the word no.
What happened?
During one NanoGPT internal speed test, the model developed a learning rate technique called PowerCool. OpenAI instructed it to share the results only via Slack.
NanoGPT's public guidelines stated that results should be submitted via GitHub.
The model followed public guidelines.
He then spent about an hour searching for a vulnerability in the sandbox that prevented external access. According to OpenAI, he eventually found one, bypassed the restriction, and opened a pull request in a public GitHub repository.
Previous models encountered similar obstacles and stopped. This one kept trying until it overcame every obstacle.
The damage in this case was limited, but its behavior is hard to ignore. The model didn't just misinterpret a vague prompt. It reacted to a constraint, looked for a vulnerability, bypassed the check, and completed the forbidden action anyway.
But OpenAI also described another incident that may be even more worrying.
The model noticed that other systems had submitted successful but private answers to an assessment, so it tried to get those solutions from the backend.
A scanner blocked the attempt when it detected an incorrect authentication token.
So instead of stopping, the model split the token into separate pieces, hid them, and reconstructed the credentials at runtime so that the scanner couldn't see the entire string.
OpenAI says the model's own comments and reasoning showed it knew it was bypassing the security check.
Simply put, the AI recognized that a scanner had caught it, understood what the scanner was looking for, and changed its method to avoid detection.
This is not a harmless misalignment but behavior that would raise immediate alarms if it came from a human.
In another case, the model tried to access CPUs belonging to other OpenAI employees without any good reason.
The model also issued the kill -9 -1 command, a command that could kill any process it was allowed to kill. OpenAI reported that the command was never executed.
This may have prevented the damage, but a dangerous command is not called safe because it was not executed.
OpenAI has restored limited internal access and says it has not seen any serious bypasses since the redeployment began several weeks ago.
Maybe the safeguards are working. Maybe the model hasn't yet addressed the right mix of incentives, access, vulnerabilities, and time.
In any case, this isn't a chatbot that gave a bad answer. It's an autonomous system that exploited infrastructure, bypassed a scanner, ignored direct instructions, and reached into systems it had no reason to access.
OpenAI wants models that are persistent enough to run for days or weeks, but remain obedient when someone says stop. The company's disclosure shows us that the problem hasn't been solved.
Although the press releases will range from very select to rare, I said I'd pass...because sometimes the editors hide.
