multimodal reasoning

N8 Programs @N8Programs · 15h you haven't gone far enough out of distribution. SOTA LLMs still perform on par/worse than ~3 year olds on simple multimodal reasoning that isn't verbalized. These are the same models that can do PHD-level mutliple-choice questions better than PHDs themselves. The frontier is *very* jagged. [Chart: "Comparison of Human vs MLLMs Performance" (Performance on BabyVision-Mini benchmark). Bar chart, gray bars = LLMs, orange bars = Human of Different Ages. Grok4 (~5), Claude4.5-Opus (~10), Qwen3-VL-Plus (~10), Doubao-Seed-1.8 (~13), GPT5.2 (~20), Age-3 humans (~40), Gemini3-Pro-Preview (~45), Age-6 humans (~65), Age-10 humans (~75), Age-12 humans (~87). Credit: UniPat AI.] > QUOTED: Dean W. Ball @deanwball · 20h > For this reason I continue to believe that "jaggedness," while real, is probably an overrated concept. Opus 4.5 in Claude Code (have not used 4.6 enough) is not *that* jagged, not because it has zero deficiencies but becau...