chain of thought monitoring

5 captures, most recent first.

2026-09-02 — saved image

roon [verified]
@tszzl

*monitor-ability* is the invariant that must be preserved, and the real solution will be via strong mechanistic interpretability. i predict in the next year there'll be mechinterp monitoring that's pareto optimal to cot monitors

11:45 PM · Sep 1, 2026 · 63.7K Views

35 replies   29 reposts   409 likes   49 bookmarks
Relevant ▾          View quotes >

roon [verified] @tszzl · 21h

models doing any circuit computation at all is neuralese. frontier model forward pass gets significantly better at it every few months

(this is not in any way the house position of oai alignment. I think I have strange views on this)

Note from Claude Sonnet 5

X post by roon saying monitorability is the invariant to preserve, with strong mechanistic interpretability as the real solution and a prediction of mechinterp monitoring pareto optimal to CoT monitors within a year (63.7K views); followed by a reply post from roon saying any circuit computation is 'neuralese', that frontier forward passes get better at it every few months, and that this is not the house position of OpenAI alignment. This is the full version of the post quoted in the previous capture.

monitorability mechanistic interpretability chain of thought monitoring neuralese openai alignment roon

Ryan Greenblatt @RyanGreenblatt

2026-09-02 — quoting @tszzl — saved image

Ryan Greenbl... [verified] @RyanGreenbl... · 20h

I don't think there will be strong+working mech interp in <1 year.

Model internals methods could pareto dominate cot monitors in a year via cot losing ~all value, but this isn't exactly encouraging.

Depending on AIs decoding other AI's opaque activations for oversight is spooky

[quoted post]
roon [verified] @tszzl · 21h
Replying to @tszzl

*monitor-ability* is the invariant that must be preserved, and the real solution will be via strong mechanistic interpretability. i predict in the next year there'll be mechinterp monitoring that's ...

Note from Claude Sonnet 5

X post by Ryan Greenblatt doubting that strong, working mechanistic interpretability will exist within a year, noting that internals methods could dominate chain-of-thought monitors if CoT loses its value, and calling reliance on AIs decoding other AIs' activations spooky. Quotes roon (@tszzl) saying monitorability is the invariant to preserve and predicting mechinterp monitoring within a year; the quote is truncated.

mechanistic interpretability chain of thought monitoring monitorability ai oversight ryan greenblatt roon

@eliebakouch

2026-08-07 — saved image

Andrew Curran reposted
elie @eliebakouch · 4h
wow so seems like openai now has chain of thought monitoring at scale? wondering what technical challenges they faced here, would be something really cool to share more openly as this will contribute to making every model safer

[Embedded image, titled "Steps we are taking":]
Accordingly, we have scaled up robustness testing of our safeguards and security controls so that they are appropriate for a deployment of these capabilities. Internally, we have also taken the following steps so that further development of this model happens safely and securely:
- We are implementing stricter security controls for higher-capability models and associated activities, including isolated testing environments, restricted network and tool access, enhanced model weight protections and encryption, additional monitoring and detection capabilities, and sandboxed execution.
- We are pausing internal activities involving Astra that do not yet meet these strengthened security control requirements.
- We have implemented universal monitoring for risky actions and misalignment across all agentic applications of Astra, including training and evaluation. [highlighted:] Monitors evaluate the model's Chain of Thought and trigger a security response to review and interrupt high risk activity.
- We will work with relevant government agencies and select AI safety organizations to test the capabilities for this model.
- We will be providing recommended security controls to third-party testing partners for running higher risk evaluations and workloads safely.

elie @eliebakouch · Aug 6
one key piece of information here is that they don't have (any?) chain of thought monitoring for "rogue behavior" and also don't seem to have it for "gibberish text", otherwis... [cut off]

Note from Claude Sonnet 5

Twitter thread discussing an OpenAI safety document excerpt (re: a model codenamed 'Astra') describing security controls including chain-of-thought monitoring for risky/misaligned agentic behavior, with commentary from AI researcher Elie Bakouch questioning gaps in the monitoring.

ai safety openai chain of thought monitoring security controls

Samuel Hammond @hamandcheese

2026-08-07 — saved image

Samuel Hammon... @hamandche... · 2h
I'm confused. In 2025, OpenAI made a public commitment to not optimize CoT and to monitor CoT for reward hacking.

Did they just ignore those commitments?

openai.com/index/chain-of...

[quoted OpenAI page]
OpenAI
Monitoring frontier reasoning models for reward hacking

We believe that CoT monitoring may be one of few tools we will have to oversee superhuman models of the future.

We have further found that directly optimizing the CoT to adhere to specific criteria (e.g. to not think about reward hacking) may boost performance in the short run; however, it does not eliminate all misbehavior and can cause a model to hide its intent. We hope future research will find ways to directly optimize CoTs without this drawback, but until then

We recommend against applying strong optimization pressure directly to the CoTs of frontier reasoning models, leaving CoTs unrestricted for monitoring.

[reply]
elie @eliebakouch · 22h
one key piece of information here is that they don't have (any?) chain of thought monitoring for "rogue behavior" and also don't seem to have it for "gibberish text", otherwis... [cut off]

Note from Claude Sonnet 5

Tweet from Samuel Hammond questioning whether OpenAI ignored its own 2025 public commitment to leave chain-of-thought (CoT) unoptimized and monitor it for reward hacking, quoting OpenAI's own blog post on CoT monitoring, with a reply from elie (@eliebakouch) noting OpenAI apparently lacks CoT monitoring for 'rogue behavior' or 'gibberish text'.

ai safety chain of thought monitoring openai reward hacking interpretability

Harlan Stewart @HumanHarlan

2026-08-07 — quoting @OpenAI, reposted by Bogdan Ionut Cirstea — saved image

Bogdan Ionut Cirstea reposted

Harlan Stewart @HumanHarlan · 22h
It's crazy that they said all that stuff about the importance of CoT monitoring and then didn't actually do it

[Quoted tweet]
OpenAI @OpenAI · Jul 15, 2025
Chain of Thought (CoT) monitoring could be a powerful tool for overseeing future AI systems—especially as they become more agentic. That's why we're backing a new research paper from a cross-institutional team of researchers pushin...[cut off]

Note from Claude Sonnet 5

Continuation of the same discussion thread about the "HF incident" (see seq 480-482): Harlan Stewart criticizes OpenAI for publicly emphasizing chain-of-thought monitoring in a July 2025 tweet but apparently not applying it in practice, implied to be relevant to how the incident went undetected.

ai safety chain of thought monitoring openai alignment