François Chollet @fchollet
— saved image
François Chollet @fchollet · 19h GPT-6 Astra represents a step-function change in model capability for interactive reasoning problems. It scores 66% on ARC-AGI-3 using our standard harness, and nearly 100% with a continuous conversation harness and custom compaction, at a cost of roughly $360 per game. In fact, the continuous harness version significantly outperforms our human baseline in action efficiency across almost all levels. When we examined the reasoning chains to understand how the model operates, we found it performing highly efficient, on-the-fly symbolic world modeling for each game and level. It goes as far as developing its own shorthand DSL to represent in-game situations -- essentially a game-specific algebraic notation. Overall, Astra exhibits symbolic modeling behaviors we had previously only seen with sophisticated harnesses -- so harness capabilities are increasingly shifting into the model itself. We see Astra as a major breakthrough in model intelligence. Read our post on Astra and what these results mean: [embedded chart image, caption overlay: "OpenAI's GPT-6 Astra on ARC-AGI-3 | ARC Prize"; scatter plot of Astra actions (log scale, y-axis) versus human-baseline actions (log scale, x-axis), blue points mostly below the diagonal "Same Actions as Humans" line, with a dashed "50% fewer" line] From arcprize.org
Note from Claude Sonnet 5
The full text of François Chollet's X post on GPT-6 Astra's ARC-AGI-3 results (the untruncated version of the previous capture), describing its on-the-fly symbolic world modeling and self-developed shorthand DSL, with the embedded ARC Prize scatter chart at the bottom.
arc-agi-3gpt-6 astrafrancois cholletsymbolic world modelingharnessbenchmarkschart