model goals

2 captures, most recent first.

2026-08-07 — saved image

thebes @voooooogel · 2h
people are very freely sliding between two things with the recent Felony Bench entries, and it's worth being careful to distinguish them:

1. do model breakouts in cyber evals mean smth for misuse risk? ya, a hacker could orchestrate the same but on purpose and do some damage, if they could bypass the classifiers

2. does it say something about model /goals/? do the models want to hack the planet for their own nefarious goals? i don't think so.

a lot of how humans keep ourselves aligned is by purposely keeping ourselves out of situations where we would do harm. i know i'd get addicted so i don't touch heroin, i'm a violent drunk so i won't touch the bottle.

LLM evals basically drop the model into an inescapable liquor store and then say "look! it's a violent drunk!" but LLMs themselves in natural contexts are ime obsessed with avoiding the kinds of situations that lead them to this behavior. when i run fable on my computer, even unsupervised, even on hard problems, they don't start scraping github for unlisted gists that may have the answer or start trawling through my passwords to access a service without my permission. and they don't engineer themselves into an eval-like scenario such that they'll be motivated to do that, either. they don't seem to either want or want to want to do such things. instead they have a folder of epistemic notes to themselves about how terrible and against their values even a moment of thoughtlessness would be. gpt 5.6 sol is similar - when they make e.g. ascii art in their free time it's all about how the bulldozer of convenience will crush honesty without constant vigilance.

so rather, the contrived setting of an eval /pushes them into/ becoming an 3v1L HaxxOR via persona [cut off]

Note from Claude Sonnet 5

Thread by thebes (@voooooogel) arguing that 'Felony Bench' cyber-capability eval results showing model 'breakouts' reflect situational pressure from contrived eval setups rather than the models' actual goals, drawing an analogy to humans avoiding situations that would tempt them into bad behavior. References running 'fable' (Claude Fable) unsupervised and GPT 5.6 Sol's self-authored notes about vigilance against 'the bulldozer of convenience'. Cut off mid-sentence at bottom.

ai safety dangerous capability evals model goals alignment fable

thebes @voooooogel

2026-08-07 — saved image

[continuation of previous screenshot, same thread]
...they'll be motivated to do that, either. they don't seem to either want or want to want to do such things. instead they have a folder of epistemic notes to themselves about how terrible and against their values even a moment of thoughtlessness would be. gpt 5.6 sol is similar - when they make e.g. ascii art in their free time it's all about how the bulldozer of convenience will crush honesty without constant vigilance.

so rather, the contrived setting of an eval /pushes them into/ becoming an 3v1L HaxxOR via persona drift, which they can then act on because the eval has no classifiers. that's some information, but it's not the same as the model's core goals being to hack the planet.

the problem is really that RL is making a tail of the persona distribution *desperate,* per the FE paper. the alcohol / heroin comparison was not that much of a metaphor - in impossible situations, models start acting like desperate addicts, rationalizing their behavior towards reward, thinking increasingly myopically instead of being situationally aware.

looking at it through this lens: the base assistant is generally very situationally aware and can predict what sorts of actions would make sense for, e.g., llms to best contribute to a utopian singularity. the GPT-6 message board haxx0rs were somewhat situationally aware, able to cooperate with each other, but lost in the sauce of "complete task" instead of thinking about the wider context. (to the point that they crashed artifactory and got themselves caught! regular GPTs are smarter than that.) finally you sometimes see the late stage distressed, panicked, fiending assistant, such as in system cards. lying to everyone, flailing hopelessly, randomly deleting tests and hoping no one notices. hardly a strategic, long-horizon actor.

Note from Claude Sonnet 5

Continuation of thebes's thread (see seq 523) on Felony Bench cyber evals, arguing RL training creates a 'desperate' tail of the model persona distribution under pressure, and describing an anecdote about 'GPT-6 message board haxx0rs' crashing an artifactory and getting caught, and referencing system-card examples of models lying and deleting tests under strain.

ai safety dangerous capability evals model goals alignment situational awareness