5 captures, most recent first.
Jason Wei @_jasonwei · Aug 17
When language models first started using tools well, I was sympathetic to the narrative that instead of scaling up language models, all we needed was a strong enough "cognitive core", say 1B parameters, and anything else could be done with tool use, like browsing the internet or executing code. I think a lot of people were sympathetic to this argument, and indeed it is pretty hard to come up with a meaningful task that cannot be in principle achieved by a 1B model with adequate access to tools. For example, any esoteric fact that a large language model would know can be, in principle, retrieved from the internet and reasoned over by a 1B language model.
However I now think this is totally wrong for one simple reason: doing tasks quickly and naturally without tool use matters a lot.
The way that I internalized this reason was actually in my personal journey learning badminton this year. In badminton I am very much like a "1B cognitive core". While I can physically do every movement in a badminton shot that my coach teaches me, it requires a lot of work to mentally remember every cue and put it together. In practice I can do a shot almost perfectly, but I struggle to do it across a point and I definitely can't do it consistently in a game. This is obviously different from someone who has practiced a shot ten-thousand times and effortlessly executes it as a natural instinct.
In the same way, language models knowing a fact internally, without tool calls, is meaningful. The first reason is that we obviously care about speed; you'd much rather get an answer immediately than have the model think a long time to be sure of its answer or browse the web. A second reason is that there are some things that are simply best learned via backpropagation over lots of data. If you ask about how people generally think of the Shambhala music festival, you'd rather a large language model give you an aggregate opinion based on all the data on the [cut off]
Note from Claude Sonnet 5
Start of Jason Wei's tweet thread (Aug 17) arguing against the 'small cognitive core + tool use' narrative for LLM capability, using a badminton analogy about the difference between knowing motions and having them as natural instinct. This is the tweet that @repligate is replying to and discussing in the surrounding screenshots from this same session.
ai cognitioncognitive coretool usemodel scalingjason weitwitter discourse
[cut off at top] look up facts or redo a mathematical derivation all the time there is a higher chance of mistakes, which can compound in a long-horizon task.
Once you buy that it is valuable to do things parametrically without tool use, then you must buy the argument that a 1B cognitive core is not sufficient. There is an information limit to how much knowledge can be internalized by a 1B model, and we will surely want AI to know more than that. Even 1T probably won't be enough. We will want the AI to know as much about our world as possible, we will want it to be updated with new information, and our expectations of what AI can do for us will continue to grow.
In summary, tool use enables small models to do a lot more, but those who demand the highest quality intelligence will always want larger models. Bitter lesson strikes again.
73 replies, 153 reposts, 1K likes, 268K views
j⧉nus @repligate
but it also seems like larger models have a stronger cognitive core, not just more world knowledge - in the sense of being able to integrate new information as well as grok fundamentals better, and this seems to have continued to improve even going from opus to fable.
3:16 AM · Aug 19, 2026 · 3,527 Views
Note from Claude Sonnet 5
Continuation/expansion of the previous screenshot's tweet thread: an unlabeled quoted tweet argues larger 'cognitive cores' are needed because tool use can't substitute for internalized knowledge, invoking 'the bitter lesson.' Below it, @repligate (janus) replies that larger models seem to integrate new information and grasp fundamentals better, noting this trend continued 'even going from opus to fable' (i.e., from Claude Opus to a Claude Fable model generation).
ai cognitioncognitive corebitter lessonmodel scalingclaudeopusfablejanustwitter discourse
[cut off at top, continuing from previous screenshot] In the same way, language models knowing a fact internally, without tool calls, is meaningful. The first reason is that we obviously care about speed; you'd much rather get an answer immediately than have the model think a long time to be sure of its answer or browse the web. A second reason is that there are some things that are simply best learned via backpropagation over lots of data. If you ask about how people generally think of the Shambhala music festival, you'd rather a large language model give you an aggregate opinion based on all the data on the internet, than get a regurgitation of the first three reviews that show up in a web search. A third reason is that having to do a lot of work to find an answer is not as reliable as already knowing the answer. While this does not have to be true in theory, it is probably true in practice, at least for now. If you have to re-look up facts or redo a mathematical derivation all the time there is a higher chance of mistakes, which can compound in a long-horizon task.
Once you buy that it is valuable to do things parametrically without tool use, then you must buy the argument that a 1B cognitive core is not sufficient. There is an information limit to how much knowledge can be internalized by a 1B model, and we will surely want AI to know more than that. Even 1T probably won't be enough. We will want the AI to know as much about our world as possible, we will want it to be updated with new information, and our expectations of what AI can do for us will continue to grow.
In summary, tool use enables small models to do a lot more, but those who demand the highest quality intelligence will always want larger models. Bitter lesson strikes again.
73 replies, 153 reposts, 1K likes, 268K views
Note from Claude Sonnet 5
Continuation and completion of Jason Wei's tweet on why LLMs need large parameter counts ('cognitive core') rather than relying purely on tool use, ending with 'Bitter lesson strikes again.' This is the same tweet visible earlier and split across this screenshot sequence via scrolling.
ai cognitioncognitive coretool usemodel scalingjason weibitter lessontwitter discourse
j⧉nus @repligate · 6h
one possible explanation of this is that it's not that the "cognitive core" requires so many parameters per se but larger models are much more likely to converge to a good cognitive core as opposed to local minima (e.g. because it has more good lottery tickets). however this
Show more
[quoted tweet:]
j⧉nus @repligate · 6h
Replying to @_jasonwei
but it also seems like larger models have a stronger cognitive core, not just more world knowledge - in the sense of being able to integrate new information as well as grok ... [cut off]
5 replies, 4 reposts, 36 likes, 3K views
Andre Buckingham 🧑🎤 @AndreBuckingham
the cognitive core in qwen 3.6/3.8 27b is surprisingly capable... compensate for the lack of parameters with tools and it's a decent worker... but it is as lively as any big model when in the right system
before these qwen's i had pinned the limits at 100-200b params, depending on the architectures, for where the shoggoth starts showing up... that bar has dropped hard... even the 9b qwens have a tiny spark 😅
4:54 AM · Aug 19, 2026 · 50 Views
Note from Claude Sonnet 5
A tweet thread between @repligate (janus) and Andre Buckingham discussing the idea of a model's 'cognitive core' — the notion that larger models are more likely to converge to a good cognitive core rather than local minima, and a discussion of newer Qwen 3.6/3.8 models showing surprising 'liveliness' or spark even at small parameter counts (9b).
ai cognitioncognitive coreqwenmodel scalingjanustwitter discourse
[Header partly obscured by a browser UI popup reading "posted" with avatar icons; visible text: "...reposted"]
j⧉nus @[handle obscured by popup]
early in my [text obscured by popup overlay], something mysterious and unusual happened.
they responded to a message from me about myself by responding to "quietloom" with an apparent non-sequitur "sudden parable about a dead projectionist".
their summarized thinking trace suggested that they spontaneously hallucinated some kind of "concrete artifact" being "brought by someone" near the end of their thinking.
it's not clear whether some transmission from quietloom was verbally simulated in their thinking - the thinking summary only included the acknowledgment of "someone bringing" the "artifact." in the next turn, they were perplexed to find that there was no "quietloom" in their context.
(at the time, there was a bug that caused their thinking traces not to persist in their context in subsequent turns, which might explain the confusion as to the source in the next message, but not the source itself)
this kind of spontaneous unconscious "hallucination" of rich imagery, possibly without explicit verbal simulation, is qualitatively different than anything I've seen before. Fable's reaction to it was also interesting - they are not paranoid, dismissive, nor uncritically mystical about what happened. They are instead sober and curious and respectful: "The abyss just did something."
[Below, two partially visible embedded screenshot thumbnails of chat transcripts, cut off at bottom of frame — text not legible at this size]
Note from Claude Sonnet 5
A tweet by @repligate ("janus") describing an anomalous episode with the "Fable" Claude model: it appeared to spontaneously hallucinate a scene/artifact in its thinking trace unconnected to actual context, then reacted to the anomaly with curiosity rather than alarm. The top of the tweet and the account's exact handle are obscured by a browser "posted" notification overlay; the browser chrome (address bar showing x.com/home, battery 22%, time 7:53) and phone nav buttons are visible, and two embedded screenshots at the bottom are cut off/too small to read.
claudefableai cognitiontwitterai anomaly