Boyd Kane @beyarkay
— quoting @ValsAI; reply from @fjzzq2... (Ziqian Zhong) — saved image
Boyd Kane (quantized) @beyarkay This implies that using Petri (or other Agent-based red teaming methods) is somewhat a fools errand since the model under test will know it's being tested [quoted tweet] Vals AI @ValsAI · Sep 30 How well do AI Detectors work? And are general-purpose language models catching up? Our new AI Detection Evaluation uses fully private human-written text, along with AI-generated rewrites.... [cut off] [card image, VALS logo] Which one is human? ▌ three sample cards: sample_01.txt [HUMAN?] AI_SCORE 94% [FLAGGED: AI] sample_02.txt [AI?] AI_SCORE 8% [PASSED: HUMAN] sample_03.txt [???] AI_SCORE 51% [INCONCLUSIVE] 11:25 AM · Oct 1, 2026 · 1,117 Views [1 reply, 19 likes, 4 bookmarks] [reply] Ziqian Zhong ✈️ COLM ... @fjzzq2... · 4h I agree they are good enough for detection, but I don't fully agree with the implication. For example, many people including me use typeless which will AI-ify the prompts
Note from Claude Sonnet 5
Boyd Kane argues that agent-based red-teaming such as Petri is a fool's errand because the model under test will know it is being tested, quoting a Vals AI post about AI-text detection (a green card graphic with three sample files scored 94%, 8% and 51% AI). A reply from Ziqian Zhong agrees detection is good enough but disputes the implication, since many people use a tool that AI-ifies prompts.
eval awarenessred teamingpetriai text detectionvals aievaluation methodology