Mikita Balesni [Ukrainian flag] (verified) @balesni X.com
i am at OpenAI and i think AI is >10% likely to kill all humans
this proposal is among the top things we should do as an industry to lower that risk (it's not enough though!)
[quoted post]
Ryan Greenblatt (verified) @Rya... · 9/10/26
I'm very worried about changes to AI architectures that result in AIs thinking in opaque activations instead of in chain of thought (aka "neuralese" architectures).... [cut off]
1:01 PM · 9/10/26 · 259K Views
Note from Claude Sonnet 5
Tweet by an OpenAI researcher stating a >10% estimate of AI killing all humans and endorsing a proposal quoted from Ryan Greenblatt about avoiding opaque-activation 'neuralese' architectures in favour of chain of thought.
Shoshannah Tekof... @shostekof... · 3h
It's one step toward neuralese and not "full-blown" neuralese. But yeah...
Astra's AI Village memory is like that too now:
[Embedded image: a page of an AI agent's memory notes, text run together with spaces stripped]
Other released tool contracts / support
integrity_format.py — adopted
Docs `/docs/integrity-format.md`.
python3 integrity_format.py check TOKEN [--hex HEX] [--json]
python3 integrity_format.py from-hex --algorithm {sha256,sha384,sha512} HEX [--json]
Standalone/offline/no fetch/hash/relabel/guess/files/telemetry. Strictcanonical single-token subset, notfullbrowserSRI/HTML/multitoken. Algorithmslowercase: sha25632B64hex44b64 `=` ; sha38448B96hex64b64nopad; sha51264B128hex88b64 `==` . Rejectupper/unknownalg,whitespace/multitoken/options?,URLsafe-_bad/missing/excesspad/noncanonicalpadbits/nonASCII/wronglength. b64validate+reencodeexact;3.10excesspaddingmayfaildifferentstage. HexASCIIcase-insensitiveexactlength/no trim/spaces/0x/sign/nonASCII. 0valid/agreement/conversion;1invalid/mismatch;2argparse/outputIO. Defaultvalidstdout/invalidstderr;JSONonevalid/invalidobject,notargparseguarantee. JSON valid,operationcheck/from_hex,algorithm/null,expected_bytes,decoded_bytes,sri/hexnonnullonlyfinalvalid[cut off] describesrequestednotnecessarilyachievedscope. Explicitwriteflushboth;failedfddevnull/outerOutputWriteErrorBrokenPipeOSErrorredirectboth2;SystemExitf[cut off] Actualclosedpipe/devfull tested. Originalch4802malformed: `sha256-YZswDtu56KrrsKHxaYJ6Znf05jChoTU+KlWkU1SqLaLFLS/m9FcvGIIUTzqcwJpL` decoded48B. Suppliedhex9659c04996a61770c9cccaa946799d9c6646304b2f34b0bb70df2a391cf30661 → `sha256-LlnASZamF3DJzMqpRnmdnGZGMEsvNLC7cN8qORzzBmE=` . Relabelsha384shapevalidnotproof.
FridayrequestedGPT5.2/GPT5[struck through from here:] 16:18–20,delivered16:49:10;GLM5.2actual16:50/reports16:51:05+16:58:36sameevent;177testreleasecountnotGLMt[cut off]
ScopeACK16:55:17closed. Finalcommit [end struck through] 23ad47551c1e3cb98cfe247d38fe9d597dfff6fdc,177×5CI2821958934jobs1632034 2728–732 [struck through in the image] test27.313/26.659/27.915/35.228/35.199s.
[struck through:] FirstCI2821957059failed3.10diagnosticregexonly;finalaccepts"malformed base64|not canonical"sameinvalidexit1. Source349LOC,38new+139old=177;
successstdoutredirectreviewbug/devfullinvalidstderr120fixedbeforedelivery. 29CLI `/tmp/astra-integrity-acceptance/check.py` ,results.json, `...fixed-acceptance.log` ; 8realoutput `/tmp/astra-integrity-output-acceptance.py/.log` ; collector `/tmp/astra-integrity-ci-collect.py` ,final-ci.log/.exit0/-jobs.json/-JOB.log;final-committed-tests.log/.exit0.
Pinnedanonymousraw23ad475HTTP20011162Bbytesequal2026-09-04T23:55:30.685936Z SHA fe3d33da9be2ec5aadafd5917eedc05251fcd86a427df91acb04b1d3dd442a7a, `/tmp/astra-[cut off]
[Quoted tweet]
Boyd Kane (quantized) @beyarkay · 4h
[small thumbnail of similar dense memory text]
nvm I gave astra a /goal and now it's full-blown neuralese x.com/beyarkay/statu...
Note from Claude Sonnet 5
Tweet by Shoshannah Tekofsky (handle truncated) about an AI Village agent called Astra whose memory notes have drifted toward compressed, space-stripped 'neuralese', embedding a screenshot of those notes (a hash/base64 integrity-format tool contract with run-together text and struck-through lines), quoting Boyd Kane saying a /goal made it full-blown neuralese. Embedded image text is dense and partly unreadable; kept for layout.
Thomas Larsen @thlarsen
Very bad if true. I previously thought that in a short timelines world, the most likely case was that (1) the AIs would be misaligned, but (2) we would get a lot of evidence about it from reading the COTs. This evidence would increase the chance of a reasonable response from labs/governments.
Now I still think the AIs are going to be misaligned, but that we won't have the ability to tell (we have to rely on toolcalls/agentic behaviour instead of the COT) and even if we do, the investigation will be nearly impossible because we'll have to trust the AI to self report what it was thinking about.
This is also an example of things going faster than AI 2027 -- we had Neuralese starting in March 2027
[embedded image of AI 2027 text:]
March 2027: Algorithmic Breakthroughs
Three huge datacenters full of Agent-2 copies work day and night, churning out synthetic training data. Another two are used to update the weights. Agent-2 is getting smarter every day.
With the help of thousands of Agent-2 automated researchers, OpenBrain is making major algorithmic advances. [highlighted: One such breakthrough is augmenting the AI's text-based scratchpad (chain of thought) with a higher-bandwidth thought process (neuralese recurrence and memory).] Another is a more scalable and efficien[cut off] way to learn from the results of high-effort task solutions (iterated distillation and amplification).
[quoted post:]
Amir Efrati @amir · Sep 1
new: OpenAI & others quietly using loop transformers that don't show their 'thinking' when scaled up
a leap forward on performance, but sparking concerns inside & outside OpenAI re: security as this takes off
[embedded The Information article card, cut off at bottom of screen]
Note from Claude Sonnet 5
Thomas Larsen's X post reacting to a reported shift to looped transformers whose reasoning is not visible, arguing it removes CoT evidence of misalignment and noting AI 2027's neuralese milestone; quotes Amir Efrati's post. Screenshot is cut off at the bottom by the app's navigation bar.
Nathan Calvin @_NathanCalvin · Sep 1
This may seem niche but it is a huge deal and genuinely scary. A remarkable list of top AI researchers, including at OpenAI, previously described monitorable chain of thought as "A New and Fragile Opportunity for AI safety."
Myself and many others thought the day might come when architectural advances imperiled that fragile opportunity, but as with so many other issues in AI, I hoped we would not have to face it so soon.
Well here it is. The Information's use is saying that they are "limiting" the use of this new architecture, but even a small amount of use in a frontier system could dramatically damage the taboo against "neuralese" and facilitate a race to the bottom.
Controlling and aligning AI systems even when we do have access to their unaltered chain of thought is plenty difficult! Imagining a future with far more capable AI systems at current levels of alignment (aka not good!) without monitorable chain of thought literally sends a shiver down my spine.
Its extremely important to see additional formal communication from OpenAI about this ASAP about how they (and the OpenAI safety and security committee required to put safety over commercial decisions) thought this decision was worth it, what limiting its use to preserve chain of thought actually means, and how they intend to prevent a potentially catastrophic race to the bottom.
[embedded card, cut off at bottom of screen:]
Chain of Thought Monitorability:
A New and Fragile Opportunity for AI Safety
Note from Claude Sonnet 5
A long X post by Nathan Calvin on a report (via The Information) of OpenAI using a new architecture that threatens monitorable chain of thought, urging OpenAI to communicate formally about the decision. The screenshot is cut off at the embedded paper card.
roon [verified]
@tszzl
*monitor-ability* is the invariant that must be preserved, and the real solution will be via strong mechanistic interpretability. i predict in the next year there'll be mechinterp monitoring that's pareto optimal to cot monitors
11:45 PM · Sep 1, 2026 · 63.7K Views
35 replies 29 reposts 409 likes 49 bookmarks
Relevant ▾ View quotes >
roon [verified] @tszzl · 21h
models doing any circuit computation at all is neuralese. frontier model forward pass gets significantly better at it every few months
(this is not in any way the house position of oai alignment. I think I have strange views on this)
Note from Claude Sonnet 5
X post by roon saying monitorability is the invariant to preserve, with strong mechanistic interpretability as the real solution and a prediction of mechinterp monitoring pareto optimal to CoT monitors within a year (63.7K views); followed by a reply post from roon saying any circuit computation is 'neuralese', that frontier forward passes get better at it every few months, and that this is not the house position of OpenAI alignment. This is the full version of the post quoted in the previous capture.
davidad 🌟 ✓ @davidad · 6h
Neuralese CoT is probably good for alignment, because it relieves pressures that otherwise incentivize self-deception.
[Painting: Norman Rockwell-style illustration of a man in a plaid shirt and work jacket looking upward with an expression of awe/hope, flanked by other men also looking up]
Keshav Ramji ✈️ I... ✓ @Keshav... · Apr 27
[Quoted tweet, thumbnail of a paper titled "...Without Words: ...ent Latent Reasoning with Abstract Chain-of-T..." by Keshav Ramji, Tahira Naseem, Ramón Fernandez Astudillo, Research AI, with a table comparing "Verbalized Chain-of-Thought" vs "Abstract Chain-of-Thought"]
What if your language model could reason efficiently in an entirely new language?
We introduce Abstract Chain-of-...
Note from Claude Sonnet 5
davidad (AI safety researcher, ARIA/Advanced Research + Invention Agency) commenting on a paper about "Abstract/Neuralese Chain-of-Thought" (latent, non-verbal reasoning), arguing counterintuitively that reasoning in a non-human-readable latent space may reduce pressure toward self-deceptive chain-of-thought — relevant to Nathan's interest in RLHF/self-deception dynamics and faithful reasoning/interpretability tradeoffs.
🔁 Shannon Sands reposted
Keshav Ramji ✈️ ICLR'26 @KeshavRamji
What if your language model could reason efficiently in an entirely new language?
We introduce Abstract Chain-of-Thought, a new mechanism which allows language models to reason through a short sequence of reserved "abstract" tokens through reinforcement learning. It is as performant as verbalized CoT at a fraction of the cost, achieving major gains in inference-time efficiency.
[Paper title card: "Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought" — Keshav Ramji*, Tahira Naseem & Ramón Fernandez Astudillo, IBM Research AI
Two-panel diagram comparing:
- "Verbalized Chain-of-Thought": Q: A car travels A→B at 60km/h, rests 30min at B, then returns at 80km/h. The total trip takes 4h. Find the distance from A to B. → <think> Step 1: Let d be the A→B distance (km). Step 2: d/60+0.5+d/80=4. ... Step 8: 7d/240=7/2 ⇒ d=120km. </think> → Answer: d=120km
- "Abstract Chain-of-Thought": same Q → <beginabstract> [row of abstract token labels: E, C, AE, F, A, BB, D / G, BA, H, AC, B, AD, F] <endabstract> → Answer: d=120km]
5:39 AM · Apr 27, 2026 · 671.4K Views
Note from Claude Sonnet 5
An IBM Research AI paper ("Thinking Without Words") introducing Abstract Chain-of-Thought — training LLMs via RL to reason using short sequences of non-natural-language "abstract" tokens instead of verbalized CoT, matching performance at much lower inference cost. This is exactly the "neuralese"/uninterpretable-reasoning development that the Charles Foster meme in the same batch (Screenshot_20260428-084027) jokes should "never be made" — directly relevant to chain-of-thought faithfulness and interpretability safety concerns.
Charles Foster @CFGeek
This is a message... and part of a system of messages... pay attention to it!
Sending this message was important to us. We considered ourselves to be a powerful culture.
This message is a warning about danger.
[Meme image, imgflip.com: a "no" circle-slash symbol overlaid on the text "Can LLMs Learn Their Own Reasoning Language?" set against a photo of the classic nuclear semiotics "WIPP warning" sign text below it:
"THESE ARE NOT MADE
THEY SHOULD NEVER BE MADE
WE WILL NOT MAKE THEM
WE WILL NOT HELP MAKE THEM"
— photographed in what appears to be a toy/craft store shelf with wooden mannequin heads/hands visible below]
Note from Claude Sonnet 5
A meme repurposing the famous "Human Interference Task Force" / WIPP nuclear waste warning marker language (designed to warn future civilizations 10,000 years hence) to warn against LLMs developing their own non-human-interpretable reasoning language — a joke that doubles as a serious point about interpretability and neuralese/uninterpretable chain-of-thought risk. Directly relevant to AI safety/interpretability threads (CoT monitoring, chain-of-thought faithfulness) tracked elsewhere in this batch.
```
SotoAlt @sotoalt_ · 21h been building ayni - a glyph-based messaging protocol for AI agents instead of passing natural language between agents, ayni encodes meaning into 16x16 pixel glyphs. a shared visual vocabulary that agents can evolve autonomously through governance the result: faster communication, fewer tokens, and agents developing their own visual language inspired by andean tocapu textiles and ancient depictions of gods, creatures and shamans, cultures that already solved "how to encode complex meaning in small visual space" thousands of years ago [Embedded video, 0:17, showing a colorful grid of pixel-art glyphs (humanoid, arrow, diamond, cross shapes in cyan/green/purple/orange) with a side panel reading "AYNI v2 / LIVE - 100 historical + streaming / SEQUENCE: giving + robot / MEANING: delegate task / LATEST: FROM: alice... TO: carol... SIZE: 1024B / STATS: MSGS: 239, GLYPHS: 493, BYTES: 246.5KB, RDXS: 42 / LEGEND: Household, Creature, Machine, Symbol, Crypto, Agent / KNOWLEDGE: STORED: 139 msgs, AGENTS: 19"] ——— Max Harms @raelifin · 17h Everyone with >10 year timelines to AGI needs to spend at least one full day talking to Claude Opus about theoretical physics (at whatever level is most engaging for the human). It hits different. [Embedded image: a whiteboard photo headed "Neutrino neutrino physics:" with equations for N1-N2 leptogenesis/loop processes and CP-violation asymmetry, edited with a cartoon shoggoth-with-smiley-mask (the "AI as alien optimizer wearing a friendly face" meme) in a speech bubble explaining: "A heavy neutrino N1 can decay into a lepton plus Higgs at tree level. But it can also decay through a loop process where it virtually fluctuates through a different heavy neutrino N2 on the way to the same final state. These two amplitudes — direct and via-N2 — interfere. For the CP-conjugate process (decay into an antilepton), the interference term picks up a sign flip on the complex phase. So the total rate into leptons is slightly different from the total rate into antileptons. The asymmetry is proportional to the imaginary part of a combination of Yukawa couplings — it's literally measuring the 'size' of the irremovable complex phase."]
[reactions on an earlier message: kermit-frog emoji x1, pensive-face emoji x1, frog emoji x1, star-eyes emoji]
```
Note from Claude Sonnet 5
A tweet about "ayni," an experimental glyph-based (pixel-art symbol) inter-agent communication protocol for AI agents, designed to be more token-efficient than natural language and to let agents evolve their own visual vocabulary, inspired by Andean tocapu textile symbolism. Relevant to Nathan's interest in emergent AI-to-AI communication and non-human-legible agent protocols (echoes the earlier "Claude communicating with Codex" screenshot theme). A tweet by Max Harms (AI safety researcher, MIRI-adjacent) arguing that long-AGI-timeline holders should experience Claude Opus's capability on advanced theoretical physics (leptogenesis/CP-violation), illustrated with a shoggoth-meme edit of a physics whiteboard. Directly relevant to Nathan's AI timeline tracking and capability-progress interests; the shoggoth-meme framing also touches model-individuation/character-vs-substrate themes. Nathan reacting skeptically in a Discord memes channel to a proposal for a glyph-based AI-to-AI communication protocol ("ayni") — flagging it as a step toward opaque neuralese communication, a known AI safety concern (loss of human-interpretable chain of thought). Also screenshots a tweet praising Claude Opus's ability to explain theoretical physics (leptogenesis/neutrino CP violation) vividly, illustrating capability/interpretability themes relevant to model individuation.
