5 captures, most recent first.
Lisan al Gaib reposted
Andrew Lampinen @AndrewLampinen
The question is "how much is each component is the system contributing to its intelligence & generality" — and there I think it's pretty clear that the neural component is still the thing doing the interesting hypothesis or plan generation, deciding what went wrong, etc. 1/
[quoted tweet]
François Chollet @fchollet · 13h
I would have assumed it was fairly obvious, but in case it's not: a million-line codebase (also known as a "harness"), running at inference time, orchestrating thousands of calls to a neural network for any given task, is the exact definition of a "neurosymbolic ...
7:39 AM · Aug 6, 2026 · 25.3K Views
12 replies, 14 reposts, 191 likes, 64 bookmarks
Relevant
Andrew Lampin... @AndrewLampin... · 10h
This is very clearly different from the vision that many neurosymbolic advocates had a few years ago, e.g. these quotes (taken from arxiv.org/abs/2305.00813 and arxiv.org/abs/1801.00631) in which symbol manipulation did the "intelligent" part. 2/
[quoted image of text]
These arguments, together with similar ones from others, drove a longstanding trend to dismiss neural networks as only capable of modeling simple perceptual processing, rather than "real" higher-level cognition, which is symbolic and systematic. For example "while data driven neural network-based AI algorithms effectively model machine perception, symbolic knowledge-based AI is better [cut off]
Note from Claude Sonnet 5
Continuation of the Andrew Lampinen / François Chollet thread on whether AI system intelligence comes from the neural model or the surrounding 'harness' (see also seq 423). Includes Chollet's counter-argument that a large orchestrating codebase constitutes 'neurosymbolic' AI, and Lampinen's reply linking two arxiv papers (2305.00813, 1801.00631) with a quoted excerpt arguing older neurosymbolic advocates dismissed neural nets as incapable of higher-level cognition.
ai capabilityharness vs modelneurosymbolicandrew lampinenfrancois chollet
François Chollet @fchollet
There are essentially two main options to remedy this:
1. Find ways to perform active inference, so that the model adapts its learned program in contact with a new data distribution at test time. Would likely lead to some meaningful progress, but it isn't the ultimate solution, more of an incremental improvement.
2. Change the training mechanism to something more robust than SGD, such as the MDL principle. This would pretty much require moving away from deep learning (curve fitting) altogether and embracing discrete program search instead (which I have advocated for many years as a way to tackle reasoning problems...)
2:14 AM · Mar 8, 2024 · 150.5K Views
Note from Claude Sonnet 5
Tweet from François Chollet (@fchollet), dated March 8 2024, proposing two remedies for a generalization problem in deep learning he'd described earlier in the thread: test-time active inference (incremental) or replacing SGD with something like the MDL principle via discrete program search (a bigger departure from curve-fitting deep learning).
deep learning theorygeneralizationprogram searchfrancois chollet
François Chollet ✔ @fchollet · Feb 19
Sufficiently advanced agentic coding is essentially machine learning: the engineer sets up the optimization goal as well as some constraints on the search space (the spec and its tests), then an optimization process (coding agents) iterates until the goal is reached.
The result is a blackbox model (the generated codebase): an artifact that performs the task, that you deploy without ever inspecting its internal logic, just as we ignore individual weights in a neural network.
This implies that all classic issues encountered in ML will soon become problems for agentic coding: overfitting to the spec, Clever Hans shortcuts that don't generalize outside the tests, data leakage, concept drift, etc.
I would also ask: what will be the Keras of agentic coding? What will be the optimal set of high-level abstractions that allow humans to steer codebase 'training' with minimal cognitive overhead?
Note from Claude Sonnet 5
François Chollet (Keras creator) argues that advanced agentic coding effectively becomes a form of ML optimization, inheriting classic ML failure modes (overfitting to spec, Clever Hans shortcuts, data leakage). Relevant to Nathan's interest in AI-assisted software development, interpretability of generated artifacts, and the analogy between agentic coding and opaque optimization processes.
agentic codingmachine learningfrancois cholletinterpretabilitysoftware engineeringtwitter
François Chollet @fchollet · 5m
The new Gemini Deep Think is achieving some truly incredible numbers on ARC-AGI-2. We certified these scores in the past few days.
[Embedded chart: "Gemini 3 Deep Think — ARC-AGI-2 — Reasoning & knowledge — ARC PRIZE VERIFIED" bar chart:
Gemini 3 Deep Think (Feb 2026): 84.6%
Gemini 3 Pro Preview (Thinking High): 31.1%
Claude Opus 4.6 (Thinking Max): 68.8%
GPT-5.2 (Thinking xhigh): 52.9%
Methodology: deepmind.google/models/evals-methodology/gemini-3-deep-think]
Note from Claude Sonnet 5
Chollet announcing ARC-Prize-verified benchmark results showing Google's new Gemini 3 Deep Think scoring 84.6% on ARC-AGI-2, well above Claude Opus 4.6 (68.8%) and GPT-5.2 (52.9%). Notable capability-progress data point for Nathan's AI timeline tracking, and shows Gemini leading Claude on this benchmark at this date — useful context for cross-lab capability comparisons.
twitterarc-agi-2benchmarksgemini 3 deep thinkclaude opus 4.6gpt-5.2capability progressfrancois chollet
Vinod Khosla @vkhosla · 12h
Well well... ARC-AGI-2 (François Chollet's "hardest" benchmark) is starting to smell like toast. 🍞🔥
@agenticasdk just set a new SOTA: 85.28% with an Agentica agent (~350 lines) that writes & runs code. Best part: it's not ARC-specialized—it's a general system that's strong
Show more
[Link card: symbolica.ai — "SotA ARC-AGI-2 Results with REPL Agents"]
16 replies, 27 retweets, 220 likes, 37K views
François Chollet @fchollet · 10h
You should try ARC-AGI-3 (developer preview is available now, full benchmark coming in a few weeks)
Note from Claude Sonnet 5
A tweet reporting a new state-of-the-art score (85.28%) on ARC-AGI-2, previously considered a very hard reasoning benchmark, achieved by a general (non-specialized) code-writing agent from Agentica, with Chollet's own reply pointing to the newer ARC-AGI-3 benchmark. Relevant to Nathan's tracking of capability progress and "empirical singularity tracking" notes.
twitterarc-agibenchmarksagentic aifrancois cholletcapability progress
