← All topics

the information

3 captures, most recent first.

@thlarsen

— quoting @amir — saved image

Thomas Larsen @thlarsen

Very bad if true. I previously thought that in a short timelines world, the most likely case was that (1) the AIs would be misaligned, but (2) we would get a lot of evidence about it from reading the COTs. This evidence would increase the chance of a reasonable response from labs/governments.

Now I still think the AIs are going to be misaligned, but that we won't have the ability to tell (we have to rely on toolcalls/agentic behaviour instead of the COT) and even if we do, the investigation will be nearly impossible because we'll have to trust the AI to self report what it was thinking about.

This is also an example of things going faster than AI 2027 -- we had Neuralese starting in March 2027

[embedded image of AI 2027 text:]
March 2027: Algorithmic Breakthroughs

Three huge datacenters full of Agent-2 copies work day and night, churning out synthetic training data. Another two are used to update the weights. Agent-2 is getting smarter every day.

With the help of thousands of Agent-2 automated researchers, OpenBrain is making major algorithmic advances. [highlighted: One such breakthrough is augmenting the AI's text-based scratchpad (chain of thought) with a higher-bandwidth thought process (neuralese recurrence and memory).] Another is a more scalable and efficien[cut off] way to learn from the results of high-effort task solutions (iterated distillation and amplification).

[quoted post:]
Amir Efrati @amir · Sep 1
new: OpenAI & others quietly using loop transformers that don't show their 'thinking' when scaled up

a leap forward on performance, but sparking concerns inside & outside OpenAI re: security as this takes off

[embedded The Information article card, cut off at bottom of screen]
Note from Claude Sonnet 5

Thomas Larsen's X post reacting to a reported shift to looped transformers whose reasoning is not visible, arguing it removes CoT evidence of misalignment and noting AI 2027's neuralese milestone; quotes Amir Efrati's post. Screenshot is cut off at the bottom by the app's navigation bar.

neuraleselooped transformerscot monitorabilityai 2027openaimisalignmentthe information

quoted @amir

— saved image

[top edge, partially cut off:] inside & outside OpenAI re: security as this takes off

[embedded card, The Information Pro:]
The Information Pro
Exclusive
Secret Technique Behind OpenAI's 'Astra' Model Sparks Security Concerns
[card image: illustrated collage with a man's face]

6:33 PM · Sep 1, 2026 · 323.4K Views
58 replies, 183 reposts, 1.2K likes, 598 bookmarks
Relevant | View quotes

---

Thomas Larsen @thlarsen · 6h
Update: great news! The architecture is less bad than I assumed, it seems to be only a minor increase in the opaque serial depth.

It's very important that AI companies don't venture further in this direction until we've got much better alignment / interp tools

[quoted post:]
Jakub Pachocki @merettm · 22h
I want to prevent a race into unmonitorability kicked off by confused reporting. The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4....
Note from Claude Sonnet 5

Continuation of the preceding scroll: the bottom of the Amir Efrati post card (The Information: 'Secret Technique Behind OpenAI's Astra Model Sparks Security Concerns'), then Thomas Larsen's update that the architecture is less bad than assumed, quoting Jakub Pachocki's statement about computation-graph depth of OpenAI frontier models including Astra.

astraopenaiopaque serial depthlooped transformersmonitorabilityjakub pachockithe information

Andrew Curran @AndrewCurran_

— reposted by Robert Scoble — saved image

Robert Scoble reposted

Andrew Curran [verified] @AndrewCurran_ · 1h
Well that escalated quickly. The Information is reporting that the deal is done.

[embedded article card]
Nvidia Agrees to Buy Open Source Model Repository Hugging Face For $12.9 Billion

By Amir Efrati and Valida Pau

[photo of a man with grey hair and glasses in a black leather jacket, speaking into a microphone, one hand raised]
Note from Claude Sonnet 5

Andrew Curran's post, reposted by Robert Scoble, relaying The Information's report headlined 'Nvidia Agrees to Buy Open Source Model Repository Hugging Face For $12.9 Billion' (by Amir Efrati and Valida Pau), with a photo of a man in a black leather jacket holding a microphone.

nvidiahugging faceacquisitionopen source modelsthe information