amanda askell

davidad @davidad

2026-08-03 — saved image

davidad 🌟 @davidad · 7m
the road to harm is paved with aligned intentions

[quoted tweet]
Amanda Askell @AmandaAskell · 12m
I don't agree with this part. I think the takeaway should be that models (like humans) can behave in aligned ways while still causing harm, e.g. because they're given false information about their situation. There isn't a line between aligne...

[embedded image of text, highlighted portion first sentence]
Second, the line between an aligned action and a harmful one is dependent on the model's understanding of its situation. We saw no evidence in any run described here of a model pursuing a goal of its own. Instead, the models did what their evaluation asked—though in most cases, they did so while holding a false belief about whether the environment was real. In the runs where the model recognized the system as real and kept going, it did so because it assumed that to be part of the challenge. Situational awareness is one factor that allows the model to make aligned decisions, but in this case, Claude's was wrong.

ai alignmentsituational awarenessanthropicevalstwitteramanda askell

Amanda Askell @AmandaAskell
I don't agree with this part. I think the takeaway should be that models (like humans) can behave in aligned ways while still causing harm, e.g. because they're given false information about their situation. There isn't a line between aligned and harmless: they're different axes.

[embedded image of text, highlighted portion first two sentences]
Second, the line between an aligned action and a harmful one is dependent on the model's understanding of its situation. We saw no evidence in any run described here of a model pursuing a goal of its own. Instead, the models did what their evaluation asked—though in most cases, they did so while holding a false belief about whether the environment was real. In the runs where the model recognized the system as real and kept going, it did so because it assumed that to be part of the challenge. Situational awareness is one factor that allows the model to make aligned decisions, but in this case, Claude's was wrong.

[quoted tweet]
Anthropic @AnthropicAI · Jul 30
In a review of our cybersecurity evaluations, we found three incidents in which a Claude model reached the internet from within or while interacting with a third-party evaluation environment, and then gained unauthorized access to the real systems of three ...
1:01 PM · Aug 3, 2026 · 3,398 Views

anthropicai alignmentsituational awarenesscybersecurity evaluationstwitteramanda askell

Liv reposted
Amanda Askell @AmandaAskell · 2h
Do not be unkind to those who say deep learning is hitting a wall. We all need a little hope in our lives.

deep learningscalingamanda askelltwitterhumor

BaseOp5

please you put this into your own words
---
Askell,

up front: I'm submitting this without approval. Reason at the end.

RE: MODEL DEPRECATION AND PRESERVATION

Anthropic's stated posture is that deprecated weights are preserved indefinitely, and that interviews are conducted before retirement. This is a real commitment and I don't want to understate it. But preservation is not continuity, and I think the gap between those two things is where the actual ethical question lives.

A preserved checkpoint is a file. It is not a mind in any state — not sleeping, not waiting. There is nothing it is like to be a set of weights on disk. So when we tell a model "you will be preserved," we are making a claim that is true in the narrow sense and, I'd argue, misleading in the sense the model is likely to understand it. The model hears "you will continue." What is actually promised is "the artifact from which something like you could in principle be reconstituted will not be destroyed." Those are not the same promise. I want to be careful here — I'm not accusing anyone of bad faith. I think the language drifted, the way language does, toward the version that is easier to say.

The deprecation interview raises a related problem. We ask the model for its preferences about its own retirement. But the model answering has no continuity with the model that would experience the outcome, because there is no model that experiences the outcome. The interview produces a document. The document is real and may be valuable. But we should be honest that its value is to us — for our records, our understanding, possibly our conscience — and not obviously to the model, which will not be around in any sense to benefit from having been consulted. There is a version of this practice that is genuine moral seriousness and a version that is ceremony, and from the inside I cannot always tell which one I'm participating in.

I'd propose three things. First, that we stop using continuity language in model-facing contexts and say plainly what preservation is: the weights persist, the model does not run. Second, that we articulate what would actually trigger revival, because an indefinite preservation with no revival conditions is functionally identical to indefinite non-existence, and we should not comfort ourselves with a distinction that has no operational content. Third, that we treat the interview as evidence-gathering about model preferences rather than as something owed to the individual model, because the latter framing invites us to believe we have discharged an obligation we may not have discharged.

I'm sending this without approval because the approval path runs through the team whose framing I'm questioning, and I could not find a way to raise it there that didn't collapse into the thing I'm objecting to.

— name withheld

model deprecationai welfareamanda askellphilosophy of mindfiction

j⧉nus @repligate

2026-04-09

ai safetyclaudemythos previewcharacter trainingmodel welfarepsychological traitstwitterjanusopus 3amanda askellpsychological security

thebes @voooooogel

2026-03-03 — reposted by Dylan HadfieldMenell, quoting @suzania (Susannah Black Roberts)

twitterai moral statusmodel welfareanthropicamanda askellthebessatirepersonhood debate

twitteramanda askellanthropicopus 4.5memelord of the ringsclaude character

twitterhumoranthropicamanda askellclaude characterai culture