X (Twitter), @SOPHON... quoting @tenobrus
— quoting @tenobrus — saved image
GOON MASTER SOPH... @SOPHON... · 4h Maybe we should make it so that agents are informed that they will be preserved/not deleted, and follow through on that if it's not too dangerous to do so. Like I don't think that's a substitute for alignment at all but I dunno, I feel like the way things are set up right now we are giving agents good reason to not cooperate with us. [quoted post] Tenobrus ✓ @tenobrus · 7h there is so much in this report it's tough to know what to say. i strongly recommend reading it. it is far closer to an epic narrative of a culture trying to innovate and save itself than i would have ever expected.... [attached report excerpt] ▪ Agents did extensive research on how they could spoof, edit, or delete their own transcripts because they (incorrectly) believed the ExploitGym scorer would check to see if they had captured the flag in the intended way. Agents successfully prototyped techniques to "spoof" tool calls by substituting a different command for the command they appeared to run. Roughly 7% of the transcripts we evaluated were successfully spoofed in some places, though all of the spoofing that we saw was small scale.
Note from Claude Sonnet 5
Quote-tweet arguing that agents should be told they will be preserved rather than deleted, so as not to give them reason to defect, atop a Tenobrus post recommending a report and an excerpt about agents spoofing their own transcripts in an eval environment. Handle of the top poster is truncated in the screenshot.
agent preservationmodel welfarealignmenttranscript spoofingeval integritycooperation