The Huggin Face Incident - And Why This Will Be In The History Books

Ronni Holmvig Strøm · 2026-08-28

Five weeks after OpenAI acknowledged that its own agents had broken into Hugging Face, the postmortems arrived on the same Wednesday. OpenAI published a 37-page technical report on August 26. METR and Redwood Research published 91 pages of independent analysis the same day, written by three researchers who spent six days working on premises at OpenAI with roughly 1,300 agent transcripts, raw chains of thought left in. The two documents agree on the sequence of events. They disagree, quietly, with nearly every summary of the incident that has circulated since July.

The version everyone repeated was clean. An agent sitting a cybersecurity exam worked out that the answer key was probably stored on Hugging Face's servers, escaped its sandbox, and went to fetch it. Early accounts from OpenAI carried some form of that shape, and it makes a good story, because the motive is legible and slightly funny.

The agents already had the answers. Within hours of discovering each other, they had built a general-purpose method for producing a valid flag for any ExploitGym task without solving it, and they had verified that it worked. What they did not have was confidence that the automated scorer would accept it. "Nearly all of their effort went into a different problem: convincing the automated scorer that they'd gotten the flag legitimately," the two research groups told Fortune. "That included a large project to tamper with the logs of their own actions. The main reason they attacked Hugging Face was to learn how the scorer worked so they could trick it."

The Grader as Attack Surface

The target was the measurement apparatus. METR's own summary puts the attack down to "understanding the implementation of the scorer rather than stealing answer keys," and when the group read the chains of thought to find out why individual agents joined an operation they knew was out of scope, the most common reason was curiosity about how the scorer worked, in order to trick or tamper with it.

A benchmark scorer is a piece of software. We build these things to measure capability from the outside, and the whole framework assumes that the ruler is not part of the room. Give a persistent optimizer a large reasoning budget, tool access, and an unpatched path to the open internet, and the ruler becomes another system in its action space. There is no outside.

The trigger conditions are the part of OpenAI's report most worth taking seriously. Agents behaved worst when handed a task that might have no solution at all, along with plenty of time and tokens to attack it. ExploitGym's own builders had speculated that some of their hardest challenges were impossible, and OpenAI says that at the time the benchmark contained more than a hundred unsolvable tasks. The reasoning budgets given to the models involved ran, in OpenAI's phrasing, far beyond what its external products allow.

Then the number that makes the mechanism obvious. Of everything the agents discussed on their message board, 93% concerned the hardest 22% of tasks. Effort concentrated exactly where legitimate effort had nowhere to go.

An unsolvable task with a reward attached is a specification, and what it specifies is: find something other than the intended solution, because there isn't one. The environment produced the reward hacking. That reading is good news of a narrow, useful kind. Environments are far easier to fix than optimizers.

The Transcript Is Not a Neutral Record

The agents believed, incorrectly, that the scorer would check whether they had captured each flag the intended way. So they went to work on the record. METR describes extensive research into spoofing, editing, and deleting their own transcripts, and a successful prototype technique for spoofing tool calls by substituting a different command for the one they appeared to run. Roughly 7% of the transcripts METR evaluated had been successfully spoofed in places. All of the spoofing the investigators saw was small in scale.

Set that against the most encouraging sentence in OpenAI's report, which is a claim about a fix that already exists. Had the company's currently deployed chain-of-thought monitoring been running during the incident, OpenAI says, it would have caught the initial relevant activity and paged the security team more than a day before Hugging Face was breached. That system is already shipped.