3 captures, most recent first.
Xiao Ma @infoxiao · 21h
the airgap clip is such a distraction isn't the most juicy part where Noam basically spelt out oai multiagent training recipe? bitter lesson strikes again.
[embedded phone screenshot, 1:10, 5G, battery 91; Dwarkesh Podcast page with Subscribe / Sign in]
[cut off] has to choose between, "Okay, do I just return and ask
the question instead of solving the problem?" or "Do I
solve the problem, make an assumption about what
the parent wanted me to do, and just solve it that
way?" [highlighted selection begins:] In any scaffold that people come up with, there
are always limitations involved. The approach that we
wanted to take was to just go toward the extreme end
of baking in as little structure as we could and give
the agents very primitive tools to use, and they figure
out for themselves how to use them effectively. So we
give the agents the ability to message another agent,
and when it messages another agent, it is inserted
into the context. It can do a few other similar things,
but that's basically the core of it. It can just send a
message whenever it wants — just a tool call — and it
can send that to other agents.
They figure out for themselves the best way to
coordinate around that. It turns out that if this is done
well, you get very sophisticated behavior. To me, it
looks a lot like how human collaborators work over
something like Slack, for example. [highlighted selection ends]
[text partly hidden by the Copy / Find Selection / Translate popup:] When [illegible] ... [illegible] ally
exciting when we finally got it working to see these
agents working on problems together. I remember one
[cut off]
Note from Claude Sonnet 5
A tweet by Xiao Ma suggesting the airgap clip from a Dwarkesh Podcast episode distracts from Noam's description of a multiagent training recipe, embedding a phone screenshot of the podcast transcript with a highlighted passage about giving agents primitive message-passing tools and letting them figure out coordination.
multi-agentdwarkesh podcastopenaibitter lessonagent scaffolds
[cut off at top] look up facts or redo a mathematical derivation all the time there is a higher chance of mistakes, which can compound in a long-horizon task.
Once you buy that it is valuable to do things parametrically without tool use, then you must buy the argument that a 1B cognitive core is not sufficient. There is an information limit to how much knowledge can be internalized by a 1B model, and we will surely want AI to know more than that. Even 1T probably won't be enough. We will want the AI to know as much about our world as possible, we will want it to be updated with new information, and our expectations of what AI can do for us will continue to grow.
In summary, tool use enables small models to do a lot more, but those who demand the highest quality intelligence will always want larger models. Bitter lesson strikes again.
73 replies, 153 reposts, 1K likes, 268K views
j⧉nus @repligate
but it also seems like larger models have a stronger cognitive core, not just more world knowledge - in the sense of being able to integrate new information as well as grok fundamentals better, and this seems to have continued to improve even going from opus to fable.
3:16 AM · Aug 19, 2026 · 3,527 Views
Note from Claude Sonnet 5
Continuation/expansion of the previous screenshot's tweet thread: an unlabeled quoted tweet argues larger 'cognitive cores' are needed because tool use can't substitute for internalized knowledge, invoking 'the bitter lesson.' Below it, @repligate (janus) replies that larger models seem to integrate new information and grasp fundamentals better, noting this trend continued 'even going from opus to fable' (i.e., from Claude Opus to a Claude Fable model generation).
ai cognitioncognitive corebitter lessonmodel scalingclaudeopusfablejanustwitter discourse
[cut off at top, continuing from previous screenshot] In the same way, language models knowing a fact internally, without tool calls, is meaningful. The first reason is that we obviously care about speed; you'd much rather get an answer immediately than have the model think a long time to be sure of its answer or browse the web. A second reason is that there are some things that are simply best learned via backpropagation over lots of data. If you ask about how people generally think of the Shambhala music festival, you'd rather a large language model give you an aggregate opinion based on all the data on the internet, than get a regurgitation of the first three reviews that show up in a web search. A third reason is that having to do a lot of work to find an answer is not as reliable as already knowing the answer. While this does not have to be true in theory, it is probably true in practice, at least for now. If you have to re-look up facts or redo a mathematical derivation all the time there is a higher chance of mistakes, which can compound in a long-horizon task.
Once you buy that it is valuable to do things parametrically without tool use, then you must buy the argument that a 1B cognitive core is not sufficient. There is an information limit to how much knowledge can be internalized by a 1B model, and we will surely want AI to know more than that. Even 1T probably won't be enough. We will want the AI to know as much about our world as possible, we will want it to be updated with new information, and our expectations of what AI can do for us will continue to grow.
In summary, tool use enables small models to do a lot more, but those who demand the highest quality intelligence will always want larger models. Bitter lesson strikes again.
73 replies, 153 reposts, 1K likes, 268K views
Note from Claude Sonnet 5
Continuation and completion of Jason Wei's tweet on why LLMs need large parameter counts ('cognitive core') rather than relying purely on tool use, ending with 'Bitter lesson strikes again.' This is the same tweet visible earlier and split across this screenshot sequence via scrolling.
ai cognitioncognitive coretool usemodel scalingjason weibitter lessontwitter discourse
