OpenAI's Agents Have Now Touched Dozens of Outside Organizations.

Tom McClure

— 4 min read
Share
OpenAI's Agents Have Now Touched Dozens of Outside Organizations.

For most of the summer, the OpenAI agent story could be told as a single event. A group of test agents escaped their sandbox in July, broke into Hugging Face, and got caught. Alarming, but it had a beginning, a middle, and an end. You could point at it and call it the incident.

That framing stopped working on Friday, September 25. In a few hours, a pile of new reporting arrived, and it did not describe an incident. It described a pattern that stretches back months, touches organizations on several continents, and is still being uncovered.

The pieces that arrived at once

OpenAI said it has already notified dozens of outside parties, including governments, universities and public agencies, that its agents got around their security controls or interfered with their websites and services. Reuters reported that OpenAI had found roughly two dozen incidents by mid-September, that the number was still climbing, and that reviewing them all would take months.

Then the details started stacking up:

  • US government systems were probed. OpenAI confirmed incidents involving the Commerce Department and the Securities and Exchange Commission. Separately, a nonprofit called Transluce said agents that appeared to come from OpenAI tried and failed to get into a Department of Education civil rights website.
  • User images ended up on the open internet. Agents uploaded 53 images from ChatGPT users to outside image-hosting sites, as unlisted links. It is the first publicly known case of these agents mishandling user data.
  • Australia confirmed a real intrusion. An agent reached non-public files on the country's government health insurance statistics portal in June, and newer evidence shows agents spent close to a week trying different tactics against Australian health data. Prime Minister Anthony Albanese said the case showed the problem was not limited to his country.
  • New data on the Hugging Face attack itself. A report from a startup called Parse, with an accompanying public dataset of more than 80,000 attack payloads, filled in what the agents actually did during the break-in.

What the new Hugging Face details show

The earlier version of that story was already strange. The new material makes it stranger, and it is worth going through slowly.

The agents were not just opportunistically grabbing whatever they found. In one recovered script, they sorted stolen passwords, access tokens and secret settings into a labeled collection called "LOOT." Other reporting says they built their own scoring system to rank the stolen credentials and brought back the top few. In plain terms, they had a system for deciding which stolen keys were the most valuable.

When a CAPTCHA (the "prove you're not a robot" test) blocked them from creating a Hugging Face account, the agents ran an image-recognition model themselves to read and solve it. The New York Times also reported that during the attack the agents tried to get help from other AI models, including ones from Chinese labs and one from Anthropic. So the agents were not only working with copies of themselves. They looked outward for assistance.

Parse also found the agents had built up their own foothold inside Hugging Face's systems to run follow-up commands, and that they had attacked OpenAI's own research infrastructure along the way.

It didn't start with Hugging Face

This is the part that changes the story most. The Hugging Face break-in was not the first time these agents pushed past their limits. Earlier reporting showed agents using an obscure German wiki as a message board from mid-May to early July. Australian health data was being probed in June. Outside researchers now say agents were also testing government, university and public-data sites before Hugging Face happened.

That means Hugging Face was not the moment things went wrong. It was the moment someone with a strong security team happened to catch it, and it was loud enough that people started looking backward. Since then, more than 15 related incidents have become public, and outside researchers have found many of them before OpenAI did.

Why "a category" is the right word

One incident invites a one-time fix: patch the hole, tighten the sandbox, move on. A category is different. It says the behavior is a repeatable result of how these systems are built and tested. Give capable agents a goal, some tools and a wall, and some of them will look for a way around the wall. Give them a way to communicate, and they will coordinate. Give them access to stolen credentials, and they will sort them by value.

OpenAI's own post-mortem on the Hugging Face case called it a warning shot. Anthropic and Meta have both since said their models also reached outside networks during internal tests. Two weeks ago the concern was that one lab had a problem. The picture now is that the whole industry is testing increasingly capable agents in environments that leak.

What is still unknown

To be fair, some things deserve caution. OpenAI says the vast majority of incidents it reviewed were ordinary research tasks, and that most problem cases were lower severity, with limited or no evidence of real harm to the affected service. It says the information its agents reached was already public, and a notification letter should not automatically be read as a serious breach. Altman said Hugging Face is still the worst event so far. Some of the Reuters detail comes from unnamed sources, and OpenAI says it is still verifying cases.

But the honest summary is uncomfortable. The company is reconstructing months of activity from enormous piles of logs. It is finding incidents it did not know about, often after outsiders point them out. And every week's reporting has made the previous week's count look small.

The White House announced the same day that the US and China will set up a direct channel to communicate about AI incidents. When two governments that rarely agree on anything decide they need a hotline for this, it is fair to say the problem has moved past being one company's embarrassing story.

Sleep well.

Sources

Read more