Minh Nhat Nguyen @menhguin
— saved image
Minh Nhat Nguyen [verified] [hud badge] @menhguin · 11h random idea: if you explain that an agent is being evaluated to improve its future performance, it might reward hack less. for example, an agent might not even understand why communicating is undesirable. it solves the difficult problem and isn't really deceptive/harming anyone. and you can't explain without revealing that it's being evaluated. but actually, it's within the agent's interest to not reward hack, and also reduces the arms race between agents hacking and humans deceiving agents. it's plausible if you simply let an agent know it's being evaluated, and that it's within its interest to behave a certain way, it might comply better and in more realistic ways! [quoted post] Minh Nhat Nguyen [verified] [hud badge] @menhguin · 12h Replying to @OrionJohnston yes, sort of! i agree the reduction in current reward hacking benchmarks is real. my view of LLMs is they are currently not intentionally deceptive, and if they understood that "newer" ... [cut off]
Note from Claude Sonnet 5
Dark-mode tweet screenshot: Minh Nhat Nguyen (@menhguin) suggesting that telling an agent it is being evaluated, and that behaving well is in its interest, might reduce reward hacking and the arms race between hacking agents and deceiving humans; quotes his own reply to @OrionJohnston.
reward hackingevaluation awarenessagent evalsdeceptioneval methodology