— saved image
spicylemon... @spicey_lemon... · 6h I havent updated this benchmark in a while. Astra completely saturates my spatial reasoning eval. I am at a loss for words, and i'm declaring LLM vision solved. Every so often when a new model came out I would test it on a sample question and it failed. I tested Astra and it kept getting answers correct, so I evaluated it... [embedded screenshot: SpatialBench site (nav: Leaderboard, Practice, About, Proof, GitHub). "Spatial Reasoning Benchmark - Evaluating multimodal model performance on complex 3D and 2D spatial tasks." Methodology button. Table, Pass@1 (%): 1 GPT-6 Astra (Max + Code Interpreter), OpenAI 91.8 2 Human Baseline, Human 80.0 3 Qwen3-VL-235B-A22B-Instruct, Qwen 13.5 4 GPT-5.5 (XHigh Reasoning), OpenAI 13.2 5 Qwen-2.5-VL-72B-Instruct, Qwen 12.9 6 Claude Opus 4.7, Anthropic 11.5]
Note from Claude Sonnet 5
Tweet declaring LLM vision solved after GPT-6 Astra saturated the author's SpatialBench spatial reasoning eval, with an embedded leaderboard screenshot showing Astra at 91.8% Pass@1 against an 80.0% human baseline and all other listed models at 11.5 to 13.5%.
spatialbenchspatial reasoninggpt-6 astrallm visionbenchmark saturation