@FakePsyho

Update on testing Astra's reasoning. It's above an average human player and honestly the ~99% on ARC-AGI-3 might undersell it. At this point I tested Astra+codex against around 30 obscure puzzle games. By tested I mean I watched it play and read it's notes. The agents were sandboxed, had no internet access and had no access to coding tools. Those were mostly puzzlescript games: some rule discovery, very tough levels, usually minimalistic level design, turn based and grid based. I used action batching to speed up the process. Some observations: \- Astra can do pathfinding/planning without code. It's not perfect and often has tiny mistakes, but I often get 20-30 actions in a single batch, which is on par with expert puzzle players. \- Due to those mistakes, I don't think it has code execution under the hood. The mistakes are very human-like: choosing longer paths that require less thinking, one off errors (miscalculating the number of moves required, resets a level that can be still solved) \- Insanely action efficient with rule discovery. Can effectively mix rule discovery with solving the level. This is probably thanks to hardcore action efficiency optimization for ARC-AGI-3. \- It understands the common visual language that puzzle games use, but has no problem with games that use completely different visual language. \- In easy/simple levels it often finds optimal/near-optimal solutions right away. \- For very hard games, it visibly gets lost. The drop in performance wrt the difficulty level is bigger than in expert players. \- For very hard levels, it's usually required to split the solving path into intermediate goals. Astra struggles with that (compared to expert players), but it's decent at that. \- In very hard levels, even when Astra looks like it's doing random pointless actions, it's still making slow progress. I'd expect that having many agents in parallel cooperating, would speed up the process. \- I'd expect that the performance drastically drops when the context grows. The games I've used are relatively simple (in terms of complexity) but very deep in terms of logic. Some example games in the comments --- Really cool "single level" sokoban-like Astra gave up after a few hour https://puzzlescript.net/play.html?p=fce521be3852cca725e85154d122e53f… --- Very cool and unusual mechanics Astra finished it without any problems https://puzzlescript.net/play.html?p=9eb8f8f3df4efb450b798a279eeba2e0… --- Simple rules but insanely hard levels. Astra worked for a hours and solved 6 levels which is a good result. [draxes.itch.io Vertebrae by draxes](https://t.co/dJlA6uOOs4) --- ##### Comments > **Karol @KarolCodes** · [2026-09-08](https://x.com/KarolCodes/status/2097327929331982610) > > in current meta, what is the human-favorable type game? > > > **Psyho @FakePsyho** · [2026-09-08](https://x.com/FakePsyho/status/2097337875796779068) > > > > 3d action games, long horizon, lots of things > **AI Quanting @AIQuanting** · [2026-09-08](https://x.com/AIQuanting/status/2097325088836321761) > > where it looks random but progresses is the interesting bit. whats the move count before the rule clicks? thats what splits working it out from searching. a human gets there in a few losses, they cant hold much. a big context can be way less efficient and still win > **FateOfMuffins @FateOfMuffins** · [2026-09-08](https://x.com/FateOfMuffins/status/2097383722412585294) > > I have a test on a relatively obscure puzzle game (based on what I've asked of its knowledge of the mechanics, it's not heavily trained on it) that requires a HUGE amount of context and it still requires a lot of hand holding from me, so that tracks > **Jatin Garg @jatingargiitk** · [2026-09-08](https://x.com/jatingargiitk/status/2097337156952039866) > > The longer path with less thinking part is what my coding agents do too. Write a 40-line workaround instead of a 5-line fix because the fix requires remembering something from 20 steps ago. Didn't expect it to look the same without code involved. > **Shubham Sharma | AI & Tech @editxshub** · [2026-09-08](https://x.com/editxshub/status/2097348563487674526) > > The interesting part is that it's mistakes are human like, > > which might prove that it's really reasoning in natural language rather than searching. > **Zack Swafford @zswaff** · [2026-09-08](https://x.com/zswaff/status/2097439843018330327) > > want to see the parallel agents version on the hard levels > **Keep Your Head @keepyourheadai** · [2026-09-08](https://x.com/keepyourheadai/status/2097340937408270408) > > Obscure to a human and obscure to the training set are different filters. Did any of the 30 have no writeups online? > **InsomniaLK @InsomniaLK** · [2026-09-08](https://x.com/InsomniaLK/status/2097327399163867587) > > That’s a pretty wild benchmark to clear. Watching it handle those obscure puzzles with notes must’ve been impressive to see in real-time. > **NDIZEYE @pndize** · [2026-09-08](https://x.com/pndize/status/2097325324790845471) > > The interesting part is that it's mistakes are human like, which might prove that it's really reasoning in natural language rather than searching. > **KYD @K\_Y\_D\_A\_A\_L** · [2026-09-08](https://x.com/K_Y_D_A_A_L/status/2097427567296872584) > > I love the part about the mistakes being so human-like > > choosing the longer path because it requires less thinking is something an actual person would do lol