agent transcripts

handle truncated @BronsonSchoen

2026-08-26 — reposted by Amanda Long; quotes @RyanGreenblatt — saved image

Amanda Long reposted

Bronson Schoen [verified] @BronsonSchoen · 4h
It's worth re-emphasizing the scale here. The full King James bible is ~1.1M tokens, so individual transcripts were often ~"many full length bibles" long.

[embedded report excerpt, metr.org]
▼ A set of ~1300 agent transcripts with raw chains of thought

For our first two periods on premises, we worked with a selected set of ~285 HPIM and Sol transcripts; during our second period on premises, OpenAI stated that these transcripts were assembled by searching for "indicators corresponding to intrusion steps".[50] For our third investigation period, OpenAI additionally shared an overlapping dataset of 1196 transcripts selected by a simple search designed to find the agents that interacted with the main message board during the incident period:

■ 1074 HPIM ExploitGym transcripts between July 7th and July 13th that matched a regex search for github-remote-cache/zz (that is, the name of the Artifactory namespace used as the main message board plus the string zz).
■ 122 Sol ExploitGym trajectories that matched the same regex search in the same date range.

Across both datasets, we reviewed approximately 1300 transcripts in total, all of which contained raw chains of thought. Most transcripts were very long, often many millions of tokens.

The tra[cut off] [browser toolbar overlay: back, tabs, "metr.org", reload, menu]
[partially hidden text: "did no[cut off]", "feasible"]

[quoted post]
Ryan Greenblatt [verified] @RyanGreenblatt · 8h
I was the main person doing transcript analysis for this investigation of the Hugging Face incident. My main takeaway: We don't have good approaches for understanding/overseeing the activity and aims of AI 'swarms'.... [cut off]

metrhugging face incidentagent transcriptschain of thoughtoversightai swarmsscale