@_rajanagarwal

**rajan agarwal** @\_rajanagarwal [2026-03-29](https://x.com/_rajanagarwal/status/2038269553788981442) i’ve also been thinking about muon for RL post-training! haven’t seen much direct work on it & math is hard my intuition behind muon is that weight matrices are linear operators & treating step size as how much a weight update can change a layer’s outputs --> leads to an orthogonalized update. this is intuitive, keeping the gradient’s singular directions and not singular-value magnitudes i think this matters a lot for generalized RL. in SL, every token gives dense signal, so batch gradients can have rich spectral structure. in GRPO, the signal is advantage-weighted and (sometimes) group normalized, so with large group sizes, these may be weakly informative or highly correlated, leaving a few loud directions plus a lot of noise (i believe) what got me thinking about this was @appliedcompute’s hypothesis about sample leverage, where at 10% success rate each successful rollout can be 81x more informative than a failed one w/ binary rewards!! there’s also probably a geometry argument here. i suspect that muon-trained models have flatter/higher-entropy singular spectra than adamw-trained ones, and kimi k2 reported that muon-pretrained checkpoints perform best with muon fine-tuning. probably, a muon base already lives in a geometry that muon continuation preserves better than optimizer-mismatched RL but idk if this is studied? if this is all right, then as RL runs get longer, more reward-diverse/generalized (math + code + tool use + open-ended), and larger-batch, muon is a clear winner? this might all be wrong, i spent a while trying to validate hypotheses with 5.4 pro & opus, so maybe it's just them being sycophantic. has anyone measured this directly with RL? --- **rajan agarwal** @\_rajanagarwal [2026-03-29](https://x.com/_rajanagarwal/status/2038269556058141075) as RL post-training gets more generalized/diverse and compute-intensive, maybe it starts to structurally resemble pre-training, making muon a good candidate? broad signal from many domains, generalizable learning across tasks muon's advantages in pre-training isn't necessarily specific to NTP. we can probably extract maximum learning from heterogeneous gradients in the same way --- **Boyuan (Nemo) Chen** @boyuan\_chen [2026-03-29](https://x.com/boyuan_chen/status/2038349988804141542) Gradient diversity collapses fast with GRPO at low success rates. You're basically learning from a handful of outlier rollouts, so the spectral structure is exactly where muon's orthogonalization should dominate. The Kimi K2 optimizer-geometry matching finding feels underexplored though. Has anyone measured effective rank of weight updates between muon and adamw over identical RL runs? --- **rajan agarwal** @\_rajanagarwal [2026-03-29](https://x.com/_rajanagarwal/status/2038350831112622080) yea i agree & i haven't seen studies on it, would love to tho --- **Harsh Bhatt** @harshbhatt7585 [2026-03-29](https://x.com/harshbhatt7585/status/2038285443532562537) I tried moun over AdamW with GRPO, it is far stable than that with moun. --- **rajan agarwal** @\_rajanagarwal [2026-03-29](https://x.com/_rajanagarwal/status/2038286473686872355) my intuition here is that muon probably provides strong benefits when the intention is generalized RL, but there's a chance that only works with muon bases just by the argument of loss landscapes --- **Mingta Kaivo 明塔 开沃** @MingtaKaivo [2026-03-30](https://x.com/MingtaKaivo/status/2038452617928425813) super interesting thread. we've been experimenting with different optimizers for fine-tuning audio models at audiowave and the step size sensitivity is real. curious if anyone has tried muon specifically for RLHF on smaller models (sub-7B) or if the benefits mostly show up at scale?