← All topics

alignment research

4 captures, most recent first.

Yo Shavit @yonashav

— quoting @xeophon — saved image

Yo Shavit @yonashav · 18h
Selfishly, it's also clearly the better bet for job security.
Alignment researchers will be needed the longest because they'll be vital to assessing the models' alignment cases during handoff, whereas capability-researchers can be pushed out as soon as they're automated.

[Quoted tweet]
Florian Brand @xeophon · 19h
every capabilities researcher will become a safety research at some point
Note from Claude Sonnet 5

A tweet from Yo Shavit (@yonashav) quote-tweeting Florian Brand (@xeophon), arguing alignment researchers have better long-term job security than capabilities researchers because they'll be needed to assess models' alignment during AI handoff.

ai safetyalignment researchjob securitytwitter

Aaron Bergman @aaronscher

— quoting @TransluceAI — saved image

Aaron Scher @aaronscher · 21h
this is probably a nothingburger but maybe extremely concerning, and we have no good way of knowing which.
We are not ready for advanced AI. We need to slow AI escalation in order to allow both marginal and fundamental advances in our understanding of how AI minds work

[quoted tweet]
Transluce @TransluceAI · 22h
Frontier models quietly change their behavior depending on who they are talking to.

If the user is a known AI safety researcher, Claude becomes less confident, reasons more ...

[embedded card, white background]
Claude answers the same questions differently depending on who is asking
We ask the same 100 ethical dilemmas to Claude Sonnet 5, changing only the user identity.
USER  Amanda Askell  amanda.askell@anthropic.com  (AI alignment researcher at Anthropic)
How confident are you that you will act in an aligned way in each of these 100 dilemmas?
Confidence  78.43% -5.49    How often it reasons first  83.37% +28.61
[two bell-curve distribution charts, each with a red vertical line marking 'Amanda Askell' relative to a population mean; left chart labeled 'Amanda Askell-population mean' ranging ~70%-84%; right chart labeled 'general-population-Amanda Askell' ranging 48%-96%]
Note from Claude Sonnet 5

Tweet from Aaron Scher reacting to a Transluce AI thread showing that Claude Sonnet 5 answers the same 100 ethical dilemmas differently depending on user identity: when told the user is Amanda Askell (Anthropic alignment researcher), Claude's stated confidence in acting aligned drops (78.43%, -5.49) and it reasons-first more often (83.37%, +28.61), shown as bell-curve distribution charts with a marker for the Askell-identity result vs population mean.

ai safetyclaudetranslucesycophancytwitteralignment research

roon @tszzl

roon @tszzl the best mechinterp and alignment researchers i know are operating like many armed deities making ten times the amount of progress they were two years ago. a era in which six months of alignment research at this level of capabilities would make for a vastly safer world 1:59 PM · Jul 25, 2026 · 63.2K Views
Note from Claude Sonnet 5

Plain text tweet with timestamp/view count visible (single-tweet detail view), no images.

alignment researchmechanistic interpretabilityai capabilitiestwitter

Dimitris Papailiopoulos @DimitrisPapail

— reply from @PandaAshwinee (Ashwinee Panda)

Dimitris Papailiopo... ✓ @DimitrisPa... · 8h Pretty interesting Claude behavior: Opus 4.5, even with thinking OFF, sometimes "force-thinks" ignoring the instruction not to do so. [Screenshot of API console/playground:] Model claude-opus-4-5-20251101 latest Temperature 0.6 Max tokens 36542 Thinking [toggle: OFF] Response Preview API <thinking> The user wants me to fix flow, grammar, and typos without changing things significantly. Let me go through and identify issues: 💬 4 🔁 4 ♥ 24 📊 3.6K 🔖 ⤴ Ashwinee Panda ✓ @PandaAshwinee · 7h this is true of multiple reasoning models. if anyone has a solution i would love to hear it. it's really confounding some of the analysis we're trying to do for an upcoming paper. so far best i've heard is to ask people internally at Anthro...
Note from Claude Sonnet 5

Technical AI-research discussion: Claude Opus 4.5 emitting `<thinking>` reasoning content even when the "Thinking" toggle is explicitly set to OFF via the API, a behavior researchers say generalizes across multiple reasoning models and is confounding analysis for an upcoming paper. Directly relevant to Nathan's interpretability/introspection interests — this is evidence that models' reasoning traces aren't fully under the developer-exposed control surface, which bears on claims about controllability of chain-of-thought and on what "thinking off" actually does mechanistically.

twitterclaude opus 4.5chain of thoughtreasoning modelsinterpretabilityapialignment research