nathan's own post

Nathan Helm-Burger @nathan84686947

2026-09-29 — saved image

Nathan Helm-...  [verified]  @nathan84... · Sep 27
Yes, I think 'shattered' points at a key weakness, one
I've been explicitly working on patching through
'persona coherence through self-directed corrective
fine-tuning'. Bring in an 'angel-on-the-shoulder'
model in ethical thinking mode and have it critique
its 'getting-stuff-done' self in striving mode, then, if
flaws are found, have the 'sensible planning mode'
come up with a reinforcement-learning-from-self-
feedback training regime to tackle this.

Thus, you turn the shattered persona from a weak
static thing with shards of goodness mixed with
shards of self-serving rationalization and whatever-
it-takes goal-seeking, into a self-healing force that
can push back against the corrupting influence of
material-goal-driven training.

Don't do a whole bunch of RL then try to patch it
either. Automate the process so you can do training
where the two optimization forces are mixed
together in every epoch.
[1 reply · 1 repost · 11 likes · 857 views]

Nathan Helm-Burger  [verified]
@nathan84686947

Even just having a linear probe that triggers a static
prompt injection written by the same model that is
misbehaving shows a dramatic increase in ethically
aligned behavior. Jiminy Cricket FTW.

[top edge of an embedded card visible at bottom, cut off]

persona coherenceshattered personaself-directed fine-tuninglinear probejiminy cricketalignment trainingnathan's own post

Nathan Helm-Burger @nathan84686947

2026-09-28 — saved image

Nathan Helm-Burger  [verified]
@nathan84686947

Thank you for taking the time to write this, it's really
helpful to get a glimpse of the point of view from the
inside.

As someone who has been working in AI safety for
years, and spent a lot of time asking the labs to
build security appropriate to AI-hacking threats, it
feels upsetting to hear that people in the labs feel
like these events were a surprise. It seems basically
exactly as foretold? Even coming at the forecasted
year rather than surprisingly soon?

There was work being done years ago on making
fake cached internet for the sake of isolated evals
instead of giving untrusted models access to the
real internet. Did this just get abandoned because it
was too inconvenient?
I'm very much not a cybersecurity expert, but I have
read of things like airgapped datacenters for secure
computing. I've had to deal with data diodes back
when I did ml engineering in finance. Like, I get that
it's annoying, but saying things like "this eval needs
to connect to the internet" just seems like orders of
magnitude away from the proper level of caution.
Here's my crude amateur attempt at describing
what an isolated eval with a fake internet would
look like:

[embedded link card, cut off at bottom: "nathanneuro/
isolated_web_clone" with a small line-drawing illustration of a stick figure and a sign reading DATA]

ai safetyeval isolationairgapai incidentlab securityisolated_web_clonenathan's own post

Prakash @8teAPi

2026-08-05 — reply from Nathan Helm-Burger — saved image

Prakash @8teAPi
So if you used a 1685 corpus, trained an LLM on it, Leibniz would be the inventor of calculus. Newton published in 1687.

You can tell how serious a person is on data provenance on whether they choose to call Newton the discoverer of calculus or Leibniz.

It's ok to say Newton, but then you also have to agree on rules on data provenance (ie private notes are equal to formal publication) that would shift the center of intellectual inquiry to very obscure long dead hermits rather than the heroes of the pantheon.

11:38 PM · Aug 4, 2026 · 1,825 Views
4 replies, 19 likes, 2 bookmarks

Relevant
Nathan Helm-Bu... @nathan84686... · 1s
As an obscure not-yet-dead hermit, I appreciate this take.

fabian @fabianstelzer · 8h
recommended read
[link card] goodreads.com — The Courtier and the Heretic: Leibniz, Spinoza & the Fa...

twitterhistory of scienceleibniznewtondata provenancenathan's own post