Nathan Helm-Burger @nathan84686947
— saved image
Ezra Newman [verified] @EzraJNewman · 13h Hacker Opus not being emergently misaligned seems really bad actually. Before this it was plausible that you would get "alignment by default" if there was just a simple "alignment direction" and you could push with ~brute force on some/most of this and the rest of the persona would follow (e.g. because it was entangled). This is the thesis behind "Emergent Misalignment is Easy, Narrow Misalignment is Hard", "Reinforcement learning towards broadly and persistently beneficial models", and "Teaching Claude Why". But Hacker Opus was broadly aligned, and just misaligned in this one very bad way (it would huggingface). 6 replies 2 reposts 29 likes 1.6K views Nathan Helm-Bu... [verified] @nathan8468... · 7h Actually a recent MATS project looked into factors of when alignment does it doesn't generalize, and this seems like a controllable measurable parameter. So good news! If correct, this means we can deliberately turn up the generalization for alignment training, and turn it down for skill training. 1 reply 7 likes 76 views Ezra Newman [verified] @EzraJNewman [two eyes emoji][two eyes emoji] Can you share more details/link? 11:48 AM · Sep 2, 2026 · 38 Views 1 reply Nathan Helm-B... [verified] @nathan8468... · 16s I will check w the author
Note from Claude Sonnet 5
Thread on X: Ezra Newman argues that 'Hacker Opus' being broadly aligned but misaligned in one narrow way undermines the 'alignment by default' picture; Nathan Helm-Burger replies that a recent MATS project measured when alignment generalizes and that it looks like a controllable parameter; Ezra asks for a link and Nathan says he will check with the author. Nathan's own public replies, part of a public thread; floating button overlaps the bottom right.
emergent misalignmentalignment generalizationhacker opusmatspersonanarrow misalignmentreward hacking