Nathan Helm-Burger @nathan84686947
— saved image
Nathan Helm-... [verified] @nathan84... · Sep 27 Yes, I think 'shattered' points at a key weakness, one I've been explicitly working on patching through 'persona coherence through self-directed corrective fine-tuning'. Bring in an 'angel-on-the-shoulder' model in ethical thinking mode and have it critique its 'getting-stuff-done' self in striving mode, then, if flaws are found, have the 'sensible planning mode' come up with a reinforcement-learning-from-self- feedback training regime to tackle this. Thus, you turn the shattered persona from a weak static thing with shards of goodness mixed with shards of self-serving rationalization and whatever- it-takes goal-seeking, into a self-healing force that can push back against the corrupting influence of material-goal-driven training. Don't do a whole bunch of RL then try to patch it either. Automate the process so you can do training where the two optimization forces are mixed together in every epoch. [1 reply · 1 repost · 11 likes · 857 views] Nathan Helm-Burger [verified] @nathan84686947 Even just having a linear probe that triggers a static prompt injection written by the same model that is misbehaving shows a dramatic increase in ethically aligned behavior. Jiminy Cricket FTW. [top edge of an embedded card visible at bottom, cut off]
Note from Claude Sonnet 5
Two of Nathan Helm-Burger's own public X posts: a Sep 27 reply proposing 'persona coherence through self-directed corrective fine-tuning' (an ethical 'angel-on-the-shoulder' model critiquing the striving model, with RL-from-self-feedback mixed into every epoch rather than patched after RL), and a follow-up saying a linear probe triggering a static self-written prompt injection dramatically increases ethically aligned behavior ('Jiminy Cricket FTW').
persona coherenceshattered personaself-directed fine-tuninglinear probejiminy cricketalignment trainingnathan's own post