gradient noise — Timeline

2026-08-29 — quoting @Orange41324306 — saved image

You Jiacheng @YouJiacheng · 23h
it's not RL. it's the batch size (gradient noise).
puffer.ai uses Muon on RL (not LLM RL), and it performs well.
because @jsuarez has made puffer so efficient, it can generate trillions of samples for RL.

[Quoted tweet]
Pollux9437 @Orange41324306 · Aug 28
I feel muon optm family is not a great fit for RL. maybe a good fit for OPD. Definately a great fit for SFT and pretrain.

Muon likes dense and clean gradients, and lost... [cut off]

Note from Claude Sonnet 5

Dark-mode tweet by You Jiacheng replying that Muon's poor fit for RL is about batch size and gradient noise, citing puffer.ai, quoting Pollux9437's claim that Muon suits SFT and pretraining but not RL.

muon optimizer reinforcement learning batch size gradient noise pufferlib sft pretraining