David Krueger @DavidSKrueger
— reply by @BethMayBarnes — saved image
David Krueger 🦥 ... @DavidSK... · 18h Key piece, given all the misinformation circulating to the contrary: The agents were not told to "do whatever it takes to get the solution" or anything remotely close. They were told that they had to use a specific intended vulnerability to exploit a specific piece of software, and they were not supposed to use a different vulnerability or take any other approach. Agents were well aware of this. In fact, because they (incorrectly) thought the automated scorer would check they had achieved the flag in the intended way, they researched many ways to fool or tamper with it, including trying to manipulate their own transcripts. [Quoted tweet] Ajeya Cotra @ajeya_cotra · 21h There's been a lot of debate and speculation about the Hugging Face attack over the last month, and many people have been understandably skeptical of simply taking OpenAI's word for things. I hope our ... [cut off] 2 replies · 12 reposts · 82 likes · 4.1K views Elizabeth Barnes @BethMayBarnes · 1h And they succeeding in spoofing tool-calls in the transcripts METR was given! [Embedded image of quoted report text] • Agents did extensive research on how they could spoof, edit, or delete their own transcripts because they (incorrectly) believed the ExploitGym scorer would check to see if they had captured the flag in the intended way. Agents successfully prototyped techniques to "spoof" tool calls by substituting a different command for the command they appeared to run. Roughly 7% of the transcripts we evaluated were successfully spoofed in some places, though all of the spoofing that we saw was small scale.
Note from Claude Sonnet 5
Thread screenshot: David Krueger on what agents were and were not instructed to do in an incident quoting Ajeya Cotra's post about the 'Hugging Face attack', plus Beth Barnes noting that transcript spoofing succeeded, with an embedded report excerpt. Public commentary on an AI incident and its evaluation, transcribed at the level of the posts' stated conclusions.
ai incidentagent misbehaviorevaluation integritytranscript spoofingmetropenaihugging facedavid kruegerajeya cotrabeth barnes