beren millidge

Rob Wiblin @robertwiblin · 29m: ""Just like the hypothetical treacherous-turn-AI, the treacherous-turn-human should look perfectly aligned and seem to only care about the wellbeing of humanity etc until their power is sufficiently established for them to deviate from this goal. ...From an alignment perspective, human innate motivational drives are deeply misaligned compared to existing AI constitutions." beren.io/2025-08-02-Do-..." [Embedded link card, blog post titled "Do We Want Obedience or Alignment?", posted August 2, 2025:] "One question which I have occasionally pondered is: assuming that we actually succeed at some kind of robust alignment of AGI, what is the alignment target we should focus on? In general, this question splits into two basic camps. The first is obedience and corrigibility: the AI system should execute the instructions given to it by humans and not do anything else. It should not refuse orders or try to circumvent what the human wants. The second is value-based alignment: The AI system embodies some set of ethical values and principles. Generally these values include helpfulness so the AI is happy to help humans but only insofar as this conforms to its ethical principles, otherwise the AI will refuse. Historically, the field initially mostly began with the latter – that AIs should be aligned to some conception of 'The Good' and not be slavishly obedient to what humans want. This was first conceptualized through the lens of the HHH (helpful-honest-harmless) assistant and implemented through RLHF or, more interestingly, constitutional AI using RLAIF. Constitutional AI is especially interesting and promising because it sets the alignment target and the conception of 'The Good' explicitly – instead of implicitly residing in some..." [text cut off]