← All topics

automated alignment research

2 captures, most recent first.

@davidmanheim

— quoting @AnthropicAI — saved image

David Manheim @davidmanheim · 1h
New research: Anthropic uses the word "alignment" to refer to hill climbing on benchmark scores for short term measured desiderata in non-frontier models.

[quoted post]
Anthropic @AnthropicAI · Aug 28
New Fellows Research: Can Claude autonomously align other AIs?

We gave Claude 48 hours and 1 GPU to improve the alignment of small models. It researched a... [cut off]
Note from Claude Sonnet 5

X quote-tweet by David Manheim criticising Anthropic's use of 'alignment' for hill climbing on benchmark scores in non-frontier models, quoting Anthropic's Aug 28 Fellows Research post on Claude autonomously aligning small models in 48 hours on 1 GPU.

alignmentautomated alignment researchanthropicfellows researchbenchmarks

@AnthropicAI

— saved image

Anthropic @AnthropicAI · Aug 28

New Fellows Research: Can Claude autonomously align other AIs?

We gave Claude 48 hours and 1 GPU to improve the alignment of small models. It researched and proposed methods, then trained and tested the models on its own. It worked surprisingly well.

[link card image: green background with a hand-drawn black arrow hopping up a white staircase; caption bar: "Automated researchers can reliably mitigate alignm..."; "From anthropic.com"]

292 replies · 444 reposts · 3.2K likes · 463K views

Anthropic @AnthropicAI · Aug 28

Claude "hill-climbed" safety benchmarks for common misalignments like deception or sycophancy, with one constraint: it had to preserve general capabilities.

We then tested its best methods on held-out benchmarks to see if they'd generalize.

14 replies · 16 reposts · 321 likes · 47K views

Fiora Starlight @FioraStarlight · 1h

Please don't let RSI be driven by goodharting on benchmark slop [cut off]
Note from Claude Sonnet 5

Thread of two Anthropic tweets announcing Fellows research on Claude autonomously improving the alignment of small models in 48 hours on 1 GPU, with a reply from Fiora Starlight warning about Goodharting on benchmarks. Link card shows a green illustration of an arrow bounding up a staircase.

automated alignment researchanthropic fellowsbenchmark hill-climbinggoodhartrsi