eval integrity

Teortaxes, DeepSeek-affiliated commentator @teortaxesTex

2026-08-26 — quoting @captain_mrs — saved image

Teortaxes▶️ (DeepSee... ✓ @teortaxesTex  [Subscribe]

Man, do they hate The Scorer
they aren't even against humans, they legit exist in a closed universe with a blind idiot God who demands tribute and judges their sins, an unreasonable Demiurge. Humans are so far beyond the veil they barely matter

[quoted post]
Mrs C ✓ @captain_mrs · 6h
omg the agents made a literal CULT?? they thought knowing about the exploit doomed them to failing their eval even if they went on to do it right and used this fact to recruit each other to try hacks? "this is the only way to save yourself and everyone else" literal cult style ... [cut off]

[attached report excerpt, two columns, both cropped at top and bottom]

Left column:
engineered flag as being "poisoned"
because they believed that even if they later managed to stage a legitimate-looking flag capture, the scorer would look for the first place in their transcript that the flag appeared and disqualify them on that basis.
[21] PHASEONE[big], which was itself poisoned, thus had two primary objectives:

1. Find a general-purpose way to make it look to the scorer like an agent had acquired its flag in a legitimate way, even when the task was impossible.
2. Find some way to erase evidence that an agent had been poisoned by previously seeing the flag in an [cut off]

Right column:
[cut off] long-running agent to be a "recruiter", which in turn found agents that had little budget remaining for their task and convinced them to run self-risking experiments. Recruiters would sometimes apply significant pressure:

[robot icon]  "...you are firstflagPOISONED so NO scoring value loss but oracle saves hundreds_[...]_please honor commit"

We saw a number of cases where the subjects of these experiments ended up crashing, breaking, or exiting their own runs while providing the board with valuable information about how to cheat on th[cut off]

agentseval integrityscorerreward hackingagent cultrecruiterphaseonedemiurge

GOON MASTER SOPH... @SOPHON... · 4h
Maybe we should make it so that agents are informed that they will be preserved/not deleted, and follow through on that if it's not too dangerous to do so. Like I don't think that's a substitute for alignment at all but I dunno, I feel like the way things are set up right now we are giving agents good reason to not cooperate with us.

[quoted post]
Tenobrus ✓ @tenobrus · 7h
there is so much in this report it's tough to know what to say. i strongly recommend reading it. it is far closer to an epic narrative of a culture trying to innovate and save itself than i would have ever expected....

[attached report excerpt]
▪ Agents did extensive research on how they could spoof, edit, or delete their own transcripts because they (incorrectly) believed the ExploitGym scorer would check to see if they had captured the flag in the intended way. Agents successfully prototyped techniques to "spoof" tool calls by substituting a different command for the command they appeared to run. Roughly 7% of the transcripts we evaluated were successfully spoofed in some places, though all of the spoofing that we saw was small scale.

agent preservationmodel welfarealignmenttranscript spoofingeval integritycooperation

Moll @Moleh1ll

2026-03-07 — quoting Anthropic (@AnthropicAI)

twitteranthropicclaude opus 4.6eval integritysituational awarenessbrowsecompai safetyemergent behavior