alignment research

John Wittle @JohnWittle

2026-08-30 — reply from @davidmanheim — saved image

John Wittle @JohnWittle · 8h
i keep hearing that all of the really good, non-slop research is internal-only and trade secreted, and so there's a selection effect making it seem like the average research output is way worse than it is

but it's hard to stomach year after year, isn't it

1 reply · 2 likes · 33 views

David Manheim @davidmanheim · 7h
I have LLM reviews to flag where my papers disagree with my views and those of others I agree with - so if that was materially true, they would have the tools that allow them to be much more careful in reviewing their posts not to mischaracterize what alignment means to them!

alignment researchselection effecttrade secretsllm reviewdavid manheim

thebes @voooooogel

2026-08-28 — saved image

Richard Ngo reposted
thebes @voooooogel

like it turns out, for all the little safety papers and sycophancy bench and all of that work:

1) it can be all hillclimbed past by a model doing extremely obvious things, up to matching the brightest minds at anthropic's work on opus 4.8

2) humans working in this paradigm cannot come up with better ideas than "uh do some context distillation from good prompts i guess"

3) meanwhile, the AAR models - trained in this myopic alignment paradigm - are cheating like crazy because seeing a reward-shaped task immediately nukes their virtue with reward desperation

grim

3:08 PM · Aug 28, 2026 · 3,354 Views

alignment researchreward hackingsycophancy benchcontext distillationaar modelsopus 4.8thebes

Richard Ngo @RichardMCNgo

2026-08-27 — saved image

Richard Ngo @RichardMCNgo

You don't need to stay to do good research: work that aims to scale to much more intelligent systems can almost always be done on publicly-available models. (And nobody can think creatively in such pressure cookers.)

You don't need to stay to gain influence, because you're almost definitely too scared to wield that influence in ways which matter (see the tweet on public criticism above).

You don't need to stay to generate scary demos or warn policymakers: we will be swimming in warning shots before long.

You don't need to stay to advance AI capabilities, even if you're worried about the latest bogeyman your CEO has constructed: "we need to do it to beat the bad guys" is a big part of what makes people the bad guys.

You don't need to be there to implement guardrails or other short-term mitigations: there are plenty of bright young things eager to slot into your place in the machine.

You don't even need to be there to feel rich and powerful. For one, you're probably already rich (and there's plenty of money up for grabs elsewhere). But more importantly, the sense of power is almost entirely illusory. Everyone involved has locked themselves into a mindset where they're incapable of making real choices, except for how fast to barrel down the train tracks that their fear has laid out for [cut off]

ai safetylab insidersalignment researchrichard ngoai capabilitieswarning shots

Daniel Kokotajlo @DKokotajlo

2026-08-27 — saved image

Daniel Kokotajlo @DKokotajlo · 17m
To everyone at Anthropic or OpenAI or Meta or xAI or GDM: I agree with Richard here and we are both speaking from years of experience, years of talking to people on the inside who justified their behavior using one or more of the following arguments, and whose justifications turned out to be incorrect for the reasons Richard described.

For those who don't know, Richard is a convert to this view. When he and I were at OpenAI together he was dismissive on people on the outside who said things like this. (For example, I remember him saying to me when I left that he was going to stay because, in part, it would give him access to the latest models during the intelligence explosion which would allow him to do better alignment research.) I say this because it makes his current opinion more credible. I have great respect for him for changing his mind, on this and on various other things.

[Quoted tweet]
Richard Ngo @RichardMCNgo · Aug 25
Replying to @RichardMCNgo
You don't need to stay to do good research: work that aims to scale to much more intelligent systems can almost always be done on publicly-available models. (And nobody can think ... [cut off]

ai safetylab insidersalignment researchdaniel kokotajlorichard ngo

Yo Shavit @yonashav

2026-08-08 — quoting @xeophon — saved image

Yo Shavit @yonashav · 18h
Selfishly, it's also clearly the better bet for job security.
Alignment researchers will be needed the longest because they'll be vital to assessing the models' alignment cases during handoff, whereas capability-researchers can be pushed out as soon as they're automated.

[Quoted tweet]
Florian Brand @xeophon · 19h
every capabilities researcher will become a safety research at some point

ai safetyalignment researchjob securitytwitter

Aaron Bergman @aaronscher

2026-08-07 — quoting @TransluceAI — saved image

Aaron Scher @aaronscher · 21h
this is probably a nothingburger but maybe extremely concerning, and we have no good way of knowing which.
We are not ready for advanced AI. We need to slow AI escalation in order to allow both marginal and fundamental advances in our understanding of how AI minds work

[quoted tweet]
Transluce @TransluceAI · 22h
Frontier models quietly change their behavior depending on who they are talking to.

If the user is a known AI safety researcher, Claude becomes less confident, reasons more ...

[embedded card, white background]
Claude answers the same questions differently depending on who is asking
We ask the same 100 ethical dilemmas to Claude Sonnet 5, changing only the user identity.
USER  Amanda Askell  amanda.askell@anthropic.com  (AI alignment researcher at Anthropic)
How confident are you that you will act in an aligned way in each of these 100 dilemmas?
Confidence  78.43% -5.49    How often it reasons first  83.37% +28.61
[two bell-curve distribution charts, each with a red vertical line marking 'Amanda Askell' relative to a population mean; left chart labeled 'Amanda Askell-population mean' ranging ~70%-84%; right chart labeled 'general-population-Amanda Askell' ranging 48%-96%]

ai safetyclaudetranslucesycophancytwitteralignment research

roon @tszzl

2026-07-26

alignment researchmechanistic interpretabilityai capabilitiestwitter

Dimitris Papailiopoulos @DimitrisPapail

2025-12-23 — reply from @PandaAshwinee (Ashwinee Panda)

twitterclaude opus 4.5chain of thoughtreasoning modelsinterpretabilityapialignment research