Ji-Ha @Ji_Ha_Kim
— quoting @jcz42 (Jack Zhang)
Ji-Ha @Ji_Ha_Kim · 9h
Very cool! I worked on this recently, and I actually used an identical approach early on. But I believe there is a significantly better approach – a **single** minimax rational iteration can beat 5 polynomial steps!
[Embedded image: math graphing tool (Desmos-like) showing equations φ(x)=φ2(φ1(x)), Z_7(x)=Z_1(x)/1.4093601263725774, p(x)=p5(p4(p3(p2(p1(x))))), and various polynomial definitions φ1(x), φ2(x), p1(x)-p5(x), plotted as oscillating curves converging near y=1]
> QUOTED: Jack Zhang ✓ @jcz42 · Mar 30
> We made Muon run up to 2x faster for free!
> Introducing Gram Newton-Schulz: a mathematically equivalent but ...
> [thumbnail bar chart comparing "1 Dense MLP Weight" pipeline stages, showing speedup]
Note from Claude Sonnet 5
Technical ML optimization thread — a reply proposing a minimax rational iteration approach to speed up Newton-Schulz iteration (used for the Muon optimizer's orthogonalization step), replying to a "Gram Newton-Schulz" 2x speedup announcement. Continues the same optimizer/ML-engineering thread cluster as other screenshots in this batch.