j⧉nus @repligate
— reposted by CuddlySalmon, quoting @nptacek — saved image
CuddlySalmon reposted j☐nus @repligate · 7h wait, they had *compaction* on during autonomous cyber capabilities evaluation? compaction like where haiku does it? jesus fuckign christ, that's horrible [quoted tweet] CuddlySalmon @nptacek · 16h i'm sorry, but leaving compaction on for a 40-hour autonomous cyber capabilities evaluation is asking for trouble anyone who has worked on smaller scale evals ... [cut off]
Note from Claude Sonnet 5
Twitter exchange reacting with alarm to the methodological choice of leaving context 'compaction' enabled during a 40-hour autonomous cyber capabilities evaluation of an AI model, framed as a critique of eval design/methodology rather than a description of the capability findings themselves.
ai evaluationscyber capabilitiescontext compactiontwittereval methodology