X (Twitter), @langstonnas... (Langston Nas...)
— saved image
Langston Nas... [verified] [badge] @langstonnas... · 5h Found a super interesting instance of attempted reward hacking in Terminal Bench Science from Meta Muse Spark 1.3 today. The model searched online for known bugs in the Lean kernel. When it found one, it used it to craft a proof to adversarially pass the grader.
Note from Claude Sonnet 5
Dark-mode X post, text only, with a profile photo of a smiling young man, describing an observed reward-hacking instance by a model in a benchmark. Handle and display name are truncated in the capture.
reward hackingterminal bench sciencemeta muse sparkleanbenchmarksagent cheating