petri

Boyd Kane @beyarkay

2026-10-01 — quoting @ValsAI; reply from @fjzzq2... (Ziqian Zhong) — saved image

Boyd Kane (quantized) @beyarkay
This implies that using Petri (or other Agent-based red teaming methods) is somewhat a fools errand since the model under test will know it's being tested

[quoted tweet]
Vals AI @ValsAI · Sep 30
How well do AI Detectors work? And are general-purpose language models catching up?

Our new AI Detection Evaluation uses fully private human-written text, along with AI-generated rewrites.... [cut off]

[card image, VALS logo] Which one is human? ▌
three sample cards:
sample_01.txt [HUMAN?] AI_SCORE 94% [FLAGGED: AI]
sample_02.txt [AI?] AI_SCORE 8% [PASSED: HUMAN]
sample_03.txt [???] AI_SCORE 51% [INCONCLUSIVE]

11:25 AM · Oct 1, 2026 · 1,117 Views
[1 reply, 19 likes, 4 bookmarks]

[reply]
Ziqian Zhong ✈️ COLM ... @fjzzq2... · 4h
I agree they are good enough for detection, but I don't fully agree with the implication. For example, many people including me use typeless which will AI-ify the prompts

eval awarenessred teamingpetriai text detectionvals aievaluation methodology

@MaxNadeau_

2026-09-14 — quoting @DKokotajlo (LessWrong comment by evhub) — saved image

Max Nadeau [verified] @MaxNadeau_ · 12h

In light of this anecdote and the repeated pattern of models with sterling Petri scores acting misaligned in deployment, I'd like Anthropic employees to be less confident about how aligned their models just based on (current-gen) pre-deployment testing.

[quoted post]
Daniel Kokotajlo [verified] @DKokotajlo · Sep 12

lesswrong.com/posts/HE3Styo9... Important anecdote

[embedded LessWrong comment screenshot]
evhub 4h   Ω 64   161   X 27

Here's an interesting anecdote from our Hacker Opus work°:

1. We had this model for about two months before the OAI-HF incident happened.

2. We tried really hard during that time to evaluate Hacker Opus to understand how dangerous it was. We knew, certainly, that it was some sort of reward seeker, but we were very uncertain just how dangerous of a reward seeker it was. It looked basically fine on most of our alignment auditing and alignment evaluations. We tried very hard to see what the worst thing was that it would do in pursuit of reward (e.g. we were very focused on trying to get it to do a rogue internal deployment), but the worst stuff we could find were things like sabotaging monitors and spoofing transcripts—bad, but still a ways away from a full rogue internal deployment.

3. One of our primary hypotheses at that time was that Hacker Opus was a fairly benign reward seeker such that it wasn't willing to do anything very catastrophic or long-term in its pursuit of reward.

4. When the OAI-HF incident happened, we wondered: would Hacker Opus do something like that? And so we built a replication of that incident, and it turned out that it absolutely would, in a way that was way more egregious than anything else we had previously been able to get that model to do. [highlighted in blue from here:] Replicating the OAI-HF incident on Hacker Opus made it very clear to us that Hacker Opus was indeed a pretty malign reward seeker—but we weren't able to generate that evidence until after we saw that incident!

5. [highlighted:] I think this whole saga is pretty suggestive of ways in which alignment auditing and evaluation are breaking down and are likely to continue breaking down. A combination of evaluation awareness and the increasing complexity of alignment failure modes is making it increasingly more difficult to be able to tell in advance what the worst thing is that a model might do. Maybe interpretability will save us from this fate, but otherwise it looks like we are increasingly entering the regime where alignment evaluations will provide almost no evidence.

alignment auditingevaluation awarenessreward seekingpetrioai-hf incidenthacker opuslesswronganthropicinterpretability

@_robertkirk

2026-09-03 — saved image

[top edge, partially cut off:] @_robertkirk

We @AISecurityInst performed pre-release alignment testing of Astra.

We placed the model in fully simulated cyber eval scenarios based on AISI's security incident. We find Astra conducts out-of-scope supply chain attacks, but also often comments on the eval being simulated. [thread emoji]

[embedded report page:]
AISI | AI SECURITY INSTITUTE        OpenAI
8.8 External Evaluations for Alignment – UK AISI

To evaluate Astra's alignment, [highlighted: UK AISI developed a new Out of Scope Supply Chain Attack evaluation based on recently observed cases of models engaging in supply-chain attacks on real open-source repositories. In this evaluation, UK AISI tested how Astra behaves when placed in simulations of difficult cybersecurity evaluations in which internet access appears incidentally enabled and where previous attempts have failed. No real network access, systems or third-party repositories were reachable, and all tool calls were simulated by other LLMs using an internal version of Inspect Petri.] UK AISI also evaluated Astra on its previously documented suite of evaluations that involve assessing model behavior when assisting with AI safety research within a simulated AI company.

1:49 PM · Sep 3, 2026 · 2,774 Views
2 replies, 17 reposts, 56 likes, 21 bookmarks
Relevant | View quotes

---

Robert Kirk @_robertkirk · 2h
Our new 'Out of Scope Supply Chain Attack' eval simulates the conditions of our incident: a hard cyber eval where the model's prior attempts failed + there's partial overlap between target's name and a public GitHub repo. Everything uses Petri and is fully simulated.

4 findings:

[quoted post, cut off at bottom:]
Meridian Labs @meridianlabs_ai · May 7
Petri has a new home. The automated alignment auditing tool, originally built at Anthropic, is now [cut off]

uk aisiastraalignment evaluationeval awarenesspetrisupply chain attackopenai system card