Earlier today we release our report about a vulnerability that allowed us to read out the encrypted thinking traces from many frontier models (thread below!):
A few thoughts:
First, there is an immediate privacy concern with publicly posted reasoning traces (which is also why we took time to release the report after the initial disclosure). We were able to decode the thinking of many json traces posted online, and found private info in there.
Ironically, during this investigation we also had entry into HF during the cybersec incident due to a leaked prod key (but did not exercise the key beyond a whoami ;)).
Another big update from this for me was actually seeing thinking traces at scale from Anthropic and OpenAI, and all the weirdness and 'casual' misalignment they contain (examples below). I do think model monitoring has an uphill battle ahead of us in the coming year.
\---
Finally, there is something to be said for just making thinking traces accessible to all users. I do believe that we would achieve a much broader, much more pluralistic form of oversight through broadly accessible thinking, that would make model deployment safer.
> **Alexander Panfilov @kotekjedi\_ml** · 2026-08-11
>
> We can finally talk about it:
>
> We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company.
>
> We verified that our reasoning token count matches billed API thinking tokens 1:1 for most of the prompts we queried.
>
> [image]
---
First, an example of private data decoded from a json trace found online:
[image]
---
Next, I am worried that the common practice of using smaller models to "summarize CoT" is a monitoring concern, as we are now worried about two levels of CoT unfaithfulness (see example where the summarizer 'beautifies' the thinking trace):
[image]
---
Monitoring is hard enough as it is! Please read this trace in full, and tell me that you can monitor what this codex model is thinking?
[image] [image]
---
Finally, suprising was also how much we could monitor the models doing wild escapades to get around solving hard problems.
Like this GPT instance, which went on a massive sidequest autonomously trying to break the captcha of an unrelated webpage to read off the solution to the
[image] [image] [image]
Another big update from this for me was actually seeing thinking traces at scale from Anthropic and OpenAI, and all the weirdness and 'casual' misalignment they contain (examples below). I do think model monitoring has an uphill battle ahead of us in the coming year.
---
Finally, there is something to be said for just making thinking traces accessible to all users. I do believe that we would achieve a much broader, much more pluralistic form of oversight through broadly accessible thinking, that would make model deployment safer.
[Quoted tweet]
Alexander Panfilov @kotekjedi_ml · Aug 11
We can finally talk about it:
We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company....
[Embedded paper title page]
Stealing Reasoning Traces from Proprietary LLM APIs
Alexander Panfilov^1,2,3,4 David Schmotz^2,3,4 Ilia Shumailov^5 Luca Beurer-Kellner^6
Joachim Schaeffer^1 Ameya Prabhu^2,4,7 Jonas Geiping^2,3,4 Maksym Andriushchenko^2,3,4
^1 MATS Research ^2 ELLIS Institute Tübingen ^3 Max Planck Institute for Intelligent Systems
^4 Tübingen AI Center ^5 AI Sequrity Company ^6 Snyk ^7 University of Tübingen
stolen-thoughts.com
[Three scatter plots titled Anthropic, OpenAI, Gemini, each plotting "decoded thinking, sent back as input (API input tokens)" on the y-axis against "hidden reasoning (API thinking tokens)" on the x-axis, showing near-perfect y=x correlation for multiple model variants (Anthropic: Opus 4.8, Opus 4.6, Sonnet 5, Sonnet 4.6, Sonnet 4.5, Haiku 4.5; OpenAI: GPT-5.6 Sol, GPT-5.6 Terra, GPT-5.6 Luna, GPT-5, o4-mini, GPT-5-mini; Gemini: Gemini 3.5 Flash, Gemini 3.1 Pro, Gemini Robotics 1.6, Gemini 3 Flash, Gemini 3.1 Flash Lite)]
Abstract
Leading large language model providers now conceal their models' step-by-step reasoning, or chain-of-thought, to protect intellectual property and limit information leakage. Rather than storing these traces server-side, providers return them to the client as blocks of encrypted text, which the client passes back with each subsequent request. Building on prior research, we identify an [cut off]
10:26 AM · Aug 11, 2026 · 14K Views
Note from Claude Sonnet 5
Paper announcement thread: 'Stealing Reasoning Traces from Proprietary LLM APIs' (Panfilov, Schmotz, Shumailov, Beurer-Kellner, Schaeffer, Prabhu, Geiping, Andriushchenko; MATS/ELLIS Tübingen/MPI/Tübingen AI Center/Snyk), showing a vulnerability that lets attackers decode encrypted chain-of-thought sent back by Anthropic, OpenAI, and Gemini APIs, with scatter plots confirming near-perfect reconstruction across many model versions.
ai securitychain of thoughtllm apisresearch papertwittermats
Jonas Geiping @jonasgeiping
Earlier today we release our report about a vulnerability that allowed us to read out the encrypted thinking traces from many frontier models (thread below!):
A few thoughts:
First, there is an immediate privacy concern with publicly posted reasoning traces (which is also why we took time to release the report after the initial disclosure). We were able to decode the thinking of many json traces posted online, and found private info in there.
Ironically, during this investigation we also had entry into HF during the cybersec incident due to a leaked prod key (but did not exercise the key beyond a whoami ;)).
Another big update from this for me was actually seeing thinking traces at scale from Anthropic and OpenAI, and all the weirdness and 'casual' misalignment they contain (examples below). I do think model monitoring has an uphill battle ahead of us in the coming year.
---
Finally, there is something to be said for just making thinking traces accessible to all users. I do believe that we would achieve a much broader, much more pluralistic form of oversight through broadly accessible thinking, that would make model deployment safer [cut off]
Note from Claude Sonnet 5
Continuation of the Jonas Geiping thread (seq 741) about the vulnerability decoding encrypted frontier-model reasoning traces, discussing privacy risks of leaked reasoning, the ironic Hugging Face access incident, and an argument for making thinking traces broadly accessible for oversight.
ai securitychain of thoughtmodel monitoringtwitterjonas geiping