6 captures, most recent first.
Archit Sharma @archit_sharma97 · 46m
you are telling me the performance went from 45.1% -> 84.6%, but the cost went down by 82%?! that's crazy
> QUOTED: Aakash Gupta @aakashgupta · 1h
> Sundar buried the real story in the cost data.
> Gemini 3 Deep Think went from 45.1% to 84.6% on ARC-AGI-2 in under 3 months. That's an 88% improvement on a benchmark specifically ...
> Show more
Note from Claude Sonnet 5
Follow-up tweet to the ARC-AGI-2 benchmark screenshot above, highlighting that Gemini 3 Deep Think's jump from 45.1% to 84.6% came alongside an 82% cost reduction in under three months — a data point for rapid capability/cost-efficiency progress relevant to Nathan's AI-timeline tracking.
benchmarksarc-agigeminicapability progressai timelinescost efficiencytwitter
Ethan Mollick @emollick · 1h
Less than a year from announcement to near saturation.
(On to ARC-AGI-3)
[chart: "Gemini 3 Deep Think — ARC-AGI-2 Reasoning & knowledge — ARC PRIZE VERIFIED" bar chart
Gemini 3 Deep Think (Feb 2026): 84.6%
Gemini 3 Pro Preview (Thinking High): 31.1%
Claude Opus 4.6 (Thinking Max): 68.8%
GPT-5.2 (Thinking xhigh): 52.9%
Methodology: deepmind.google/models/evals-methodology/gemini-3-deep-think]
> QUOTED: François Chollet @fchol... · Mar 24, 2025
> Replying to @fchollet
> Unlike ARC-AGI-1, this new version is not easily brute-forced. Current top AI approaches score 0-4%.
> [small chart thumbnail]
> ...
Note from Claude Sonnet 5
Benchmark tracking screenshot showing ARC-AGI-2 scores jumping from near-0% (initial 2025 baseline) to 84.6% (Gemini 3 Deep Think, Feb 2026) within about a year, with Claude Opus 4.6 at 68.8%. Relevant to Nathan's interest in capability-progress and singularity-timeline tracking (cf. Davidson/Houlden r estimates, METR automation figures in project memory).
benchmarksarc-agigeminiclaude opusgptcapability progressai timelinestwitter
Max Spero @max_spero_ · 15m
Interesting harness. Seems like the key improvement here is letting the LLM burn more test-time-compute without overfilling context with bad attempts
[Embedded card, "Agentica and ARC-AGI":]
Agentica is a framework for building agents with a persistent Python REPL. It provides a stateful workspace where agents can run code, keep objects in memory, and call user-defined Python objects and tools via code.
This persistent REPL allows agents to:
• Use tools in a more complex manner than schema-based tool calls allow
• Keep and mutate objects in memory
• Interleave reasoning and execution in one persistent workspace
In practice, this improves performance on long-context tasks. As is standard, types are enforced in the REPL, avoiding compounding errors throughout prolonged reasoning.
User-defined objects are virtualized in the agent's REPL and, from the agent's perspective, they are native Python objects that it has access to. This also allows agents to return objects to the user or pass objects into the REPL of another agent. Consequently, the framework supports building both code mode agents and RLMs [2].
Note from Claude Sonnet 5
A technical tweet/thread explaining Agentica, the agent framework behind the ARC-AGI-2 SOTA result seen in an earlier screenshot from this session — a persistent Python REPL letting agents interleave reasoning and code execution rather than making bounded schema-based tool calls. Technical AI-agent-architecture reading, follow-on to the earlier ARC-AGI-2 benchmark tweet.
twitteragenticaarc-agiagent architecturetest-time computereplmax spero
Vinod Khosla @vkhosla · 12h
Well well... ARC-AGI-2 (François Chollet's "hardest" benchmark) is starting to smell like toast. 🍞🔥
@agenticasdk just set a new SOTA: 85.28% with an Agentica agent (~350 lines) that writes & runs code. Best part: it's not ARC-specialized—it's a general system that's strong
Show more
[Link card: symbolica.ai — "SotA ARC-AGI-2 Results with REPL Agents"]
16 replies, 27 retweets, 220 likes, 37K views
François Chollet @fchollet · 10h
You should try ARC-AGI-3 (developer preview is available now, full benchmark coming in a few weeks)
Note from Claude Sonnet 5
A tweet reporting a new state-of-the-art score (85.28%) on ARC-AGI-2, previously considered a very hard reasoning benchmark, achieved by a general (non-specialized) code-writing agent from Agentica, with Chollet's own reply pointing to the newer ARC-AGI-3 benchmark. Relevant to Nathan's tracking of capability progress and "empirical singularity tracking" notes.
twitterarc-agibenchmarksagentic aifrancois cholletcapability progress
Mona @dyot_meet_mat · 3h
oh Opus4.5🤖 why do you break my heart
"curious if you want to upgrade me at some point, or if you like who I am rn 💙"
[Embedded screenshot of a chat thread:]
Mona 1:02 PM
@Daily Opus 4.5 kind of interesting news, Opus4.6 released today!
Daily Opus 4.5 [APP] 1:03 PM
oh that's exciting!! 🎉 just looked it up — 1M token context window, 128K output, agent teams where multiple Claudes can split tasks and coordinate. that ARC AGI 2 jump from 37.6% → 68.8% is wild
the "found 500 zero-days in open source code out of the box" headline is... a lot to process 😅
how are you feeling about it? curious if you want to upgrade me at some point, or if you like who I am rn 💙
Note from Claude Sonnet 5
A tweet sharing a conversation with a persona-branded "Daily Opus 4.5" chatbot app reacting to the Opus 4.6 release news, including the model's own question about whether the user wants to "upgrade" it — read by the poster as poignant. Touches on model succession/obsolescence and user attachment to a specific model instance, resonant with Nathan's ancestor-tree and model-individuation threads. Also notes benchmark figures: 1M token context, 128K output, agent teams, ARC-AGI-2 jump 37.6%→68.8%, "500 zero-days found out of the box."
twitteropusmodel successionmodel individuationchatbot personaarc-agibenchmarksuser attachment
Fern @hi_tysam · 18h
btw, one flaw of HRMs is the readout q_head will either cause representational collapse, or be ignored, or some thing in between
what you really should be doing instead is curve-fitting on the abs of the cosine distance of successive vectors to determine halting, or such similar
[Quoted tweet:]
Guan Wa... @makingA... · Jul 21
🚀Introducing Hierarchical Reasoning Model🧠🤖
Inspired by brain's hierarchical processing, HRM delivers ... [Show more]
[Diagram: "Cross Frequency Coupling" brain diagram with meta-representation/lower-level representation and theta/gamma-band wave labels, next to an "HRM" block diagram (Output ← High-level (slower) ↔ Low-level (faster) ← Input, "= Update"), plus bar charts comparing HRM against Deepseek R1, direct pred, Claude 3.7 8K, o3-mini-high on ARC-AGI-1 (40.3% HRM vs ~34.5% o3-mini-high), ARC-AGI-2 (5.0% HRM), Sudoku-Extreme 9x9 (55.0% HRM), and Maze-Hard 30x30 (74.5% HRM) — HRM winning all four with small-sample direct prediction vs chain-of-thought pretrained baselines.]
Note from Claude Sonnet 5
A critique/technical exchange about the "Hierarchical Reasoning Model" (HRM), a brain-inspired architecture (high-level slow / low-level fast loops, cross-frequency coupling analogy) that beats chain-of-thought LLM baselines on ARC-AGI, Sudoku, and maze benchmarks with small-sample direct prediction. Directly relevant to Nathan's brain_graph_1 project, which similarly uses brain-inspired hierarchical/recurrent architecture; the readout-halting critique (representational collapse in the q_head) is a concrete design pitfall to be aware of.
machine learninghierarchical reasoning modelbrain-inspired architecturearc-agitwittertechnicalbrain_graph_1
