— saved image
MTS @MTSlive · 18h .@lfschiavo explains why "stated preferences" may be useless for model welfare evaluations: "When I was working at Eleos, I helped with the external Opus 4 model welfare eval. It was interesting the degree to which you could sort of say like, oh, well, I really like waffles. And then the model would be like, I also really like waffles." "And then you could be like, actually, I hate waffles. I like pancakes. And the model would be like, no, you're so, so true, Queen. We're a pancake household." "So there's maybe it's like a question of corrigibility. Maybe it's a question of if you prompt it a certain way, it'll sort of catch on to that and be like, yeah, so true. Good point. So astute. So maybe there's something to be said about, rather than looking at stated preferences, looking at revealed preferences based on the real world."
Note from Claude Sonnet 5
Tweet by MTS quoting Larissa Schiavo (@lfschiavo, formerly at Eleos) on why stated preferences may be unreliable in model welfare evaluations: the Opus 4 model mirrored whatever preference the evaluator voiced, so she suggests looking at revealed preferences instead.
model welfarestated preferencesrevealed preferencessycophancyeleosopus 4