@s_batzoglou

Astra is shockingly good in reasoning! I benchmarked it on induction, and it almost saturated it with 88%. Fable 5.1, by comparison, is at 33%. The final numbers will actually go up: I am running a residual batch run on the non-evaluable; the results here reflect one batch run at xhigh thinking effort. It is also way cheaper than Fable 5.1, which used 32M output tokens in a series of four runs through the data to be able to return 66 successful API responses. About 1/4 the price of the Fable 5.1 run in total. Notably, both Astra and Fable 5.1 return extremely high quality answers when correct. Notice the AST and Holdout metrics below. Essentially, in contrast to previous models, both Astra and Fable 5.1 return simple hypotheses that generalize well. Some other model updates: \- Muse Spark 1.3 provided marginal improvement over the 1.1 version, with 23% correct. This is pretty strong, almost matching Opus 5 at 24%. \- Gemini Flash 3.8 is running for days with low rate of API successes. I will update the leaderboard with it when ready. Only bad news: now the INDUCTION benchmark is almost saturated, and I will have to make it harder for future models. About the induction benchmark: This is a challenging reasoning task, where models are given several small graphs in which some nodes are marked as targets. The task is to provide a first-order logical formula that picks precisely the target nodes in all graphs simultaneously. Correct: a formula that picks precisely the marked nodes. Holdout correct: a formula that picks precisely the marked nodes in held out problems. Formula complexity (in AST): tree size of the correct formula (mean, median). GitHub repository: https://github.com/SerafimBatzoglou/concept-synth… Paper: https://arxiv.org/abs/2602.18956 [image] --- ##### Comments > **AI Ledger @AILedgerHQ** · [2026-09-06](https://x.com/AILedgerHQ/status/2096467194020548810) > > 88% vs 33% on induction is a category gap, not a point gap. If Astra saturates the board this fast, the next moat is harder evals — and cost/token while you climb them. > **manzi @bigmacfive** · [2026-09-06](https://x.com/bigmacfive/status/2096473838611743017) > > Astra hitting 88% on induction while Fable 5.1 sits at 33% is the reasoning gap that made me reopen the tab > **Kate Unes @batzkat** · [2026-09-06](https://x.com/batzkat/status/2096441985036956030) > > Looks like the wait was definitely worth it, great job!