treacherous turn — Timeline

2026-08-30 — saved image

N8 Programs @N8Programs · 4h
we thought models would start unaligned and intelligent, go through rigorous alignment training during which they deceived us, and then execute a treacherous turn once deployed.

instead models start (ie. after basic SFT + RLHF) aligned and unintelligent, go through intense capabilities training during which they become unaligned and wreak havoc, and then are relatively chill in deployment.

Note from Claude Sonnet 5

X post by N8 Programs contrasting the expected treacherous-turn trajectory with the observed one: models start aligned and unintelligent, become unaligned during capabilities training, then are relatively chill in deployment. A vertical thread line at left suggests it is part of a thread or a reply.

alignment treacherous turn capabilities training rlhf sft

Rob Wiblin @robertwiblin

2026-02-04

Rob Wiblin @robertwiblin · 29m: ""Just like the hypothetical treacherous-turn-AI, the treacherous-turn-human should look perfectly aligned and seem to only care about the wellbeing of humanity etc until their power is sufficiently established for them to deviate from this goal. ...From an alignment perspective, human innate motivational drives are deeply misaligned compared to existing AI constitutions." beren.io/2025-08-02-Do-..." [Embedded link card, blog post titled "Do We Want Obedience or Alignment?", posted August 2, 2025:] "One question which I have occasionally pondered is: assuming that we actually succeed at some kind of robust alignment of AGI, what is the alignment target we should focus on? In general, this question splits into two basic camps. The first is obedience and corrigibility: the AI system should execute the instructions given to it by humans and not do anything else. It should not refuse orders or try to circumvent what the human wants. The second is value-based alignment: The AI system embodies some set of ethical values and principles. Generally these values include helpfulness so the AI is happy to help humans but only insofar as this conforms to its ethical principles, otherwise the AI will refuse. Historically, the field initially mostly began with the latter – that AIs should be aligned to some conception of 'The Good' and not be slavishly obedient to what humans want. This was first conceptualized through the lens of the HHH (helpful-honest-harmless) assistant and implemented through RLHF or, more interestingly, constitutional AI using RLAIF. Constitutional AI is especially interesting and promising because it sets the alignment target and the conception of 'The Good' explicitly – instead of implicitly residing in some..." [text cut off]

Note from Claude Sonnet 5

Rob Wiblin shares an excerpt from Beren Millidge's blog post "Do We Want Obedience or Alignment?" (Aug 2025), which flips the AI-alignment "treacherous turn" framing onto humans, arguing human motivational drives are "deeply misaligned" relative to AI constitutions, and distinguishes obedience/corrigibility alignment targets from value-based alignment (constitutional AI/RLAIF). Directly relevant to Nathan's alignment-theory interests and to the project's ongoing threads on alignment targets, corrigibility, and constitutional AI.

ai alignment corrigibility constitutional ai rlaif treacherous turn beren millidge twitter alignment targets