batch size — Timeline

2026-08-29 — quoting @Orange41324306 — saved image

You Jiacheng @YouJiacheng · 23h
it's not RL. it's the batch size (gradient noise).
puffer.ai uses Muon on RL (not LLM RL), and it performs well.
because @jsuarez has made puffer so efficient, it can generate trillions of samples for RL.

[Quoted tweet]
Pollux9437 @Orange41324306 · Aug 28
I feel muon optm family is not a great fit for RL. maybe a good fit for OPD. Definately a great fit for SFT and pretrain.

Muon likes dense and clean gradients, and lost... [cut off]

Note from Claude Sonnet 5

Dark-mode tweet by You Jiacheng replying that Muon's poor fit for RL is about batch size and gradient noise, citing puffer.ai, quoting Pollux9437's claim that Muon suits SFT and pretraining but not RL.

muon optimizer reinforcement learning batch size gradient noise pufferlib sft pretraining

Martin Marek @mrtnm

2025-08-29

Getting small batch sizes to work in bfloat16 precision can be challenging. In our recent paper on batch size, we ran all experiments in float32, but memory-constrained settings demand lower precision. Here are two tricks that we used to enable bf16 training at small batch sizes: [Chart: "Pretraining 30M model, weights dtype" — FineWeb Edu loss (y-axis, 3.6–5.0) vs Batch size (x-axis, log scale 1–1024). Three lines: BF16 (closest) [gray dashed, spikes badly around batch size 64], BF16 (stochastic) [orange dashed, tracks closely with FP32], FP32 [blue, baseline]. BF16 (closest) diverges sharply upward around batch size 64 while stochastic rounding stays close to FP32 across the whole range.] 4:16 AM · Aug 28, 2025 · 11.7K Views 💬 4 🔁 18 ❤ 156 🔖 110 ⤴ Martin Marek @mrtnm · 20h (1) We recommend using decay rates like b2=0.9999 for small batch sizes. However, bf16 only has ~2.4 decimal points of precision. Since Adafactor's state is so tiny compared to the model size, we can store it in float32 without meaningfully affecting the overall memory footprint.

Note from Claude Sonnet 5

A technical ML-training thread about a batch-size scaling paper, showing that naive ("closest") bf16 rounding badly diverges from FP32 loss curves at small batch sizes while stochastic rounding tracks FP32 closely; follow-up recommends storing optimizer state in FP32. Relevant to Nathan's own ML/training work (brain_graph_1 uses similar precision tradeoffs — FP16+per-block-scales noted in his architecture notes).

twitter machine learning bfloat16 precision batch size training optimizer adafactor