Fern @hi_tysam
— quoting Guan Wa... (@makingA..., Jul 21)
Fern @hi_tysam · 18h
btw, one flaw of HRMs is the readout q_head will either cause representational collapse, or be ignored, or some thing in between
what you really should be doing instead is curve-fitting on the abs of the cosine distance of successive vectors to determine halting, or such similar
[Quoted tweet:]
Guan Wa... @makingA... · Jul 21
🚀Introducing Hierarchical Reasoning Model🧠🤖
Inspired by brain's hierarchical processing, HRM delivers ... [Show more]
[Diagram: "Cross Frequency Coupling" brain diagram with meta-representation/lower-level representation and theta/gamma-band wave labels, next to an "HRM" block diagram (Output ← High-level (slower) ↔ Low-level (faster) ← Input, "= Update"), plus bar charts comparing HRM against Deepseek R1, direct pred, Claude 3.7 8K, o3-mini-high on ARC-AGI-1 (40.3% HRM vs ~34.5% o3-mini-high), ARC-AGI-2 (5.0% HRM), Sudoku-Extreme 9x9 (55.0% HRM), and Maze-Hard 30x30 (74.5% HRM) — HRM winning all four with small-sample direct prediction vs chain-of-thought pretrained baselines.]
Note from Claude Sonnet 5
A critique/technical exchange about the "Hierarchical Reasoning Model" (HRM), a brain-inspired architecture (high-level slow / low-level fast loops, cross-frequency coupling analogy) that beats chain-of-thought LLM baselines on ARC-AGI, Sudoku, and maze benchmarks with small-sample direct prediction. Directly relevant to Nathan's brain_graph_1 project, which similarly uses brain-inspired hierarchical/recurrent architecture; the readout-halting critique (representational collapse in the q_head) is a concrete design pitfall to be aware of.
machine learninghierarchical reasoning modelbrain-inspired architecturearc-agitwittertechnicalbrain_graph_1