Simo Ryu

Simo Ryu ✔ @cloneofsimo · Jul 7 Sell the SHOVEL BLADE to shovel maker with 99% profit margin [Quoted tweet:] Jukan @ ICML ✔ [icon] @jukan05 · Jul 6 OFFICIAL: SAMSUNG OVERTAKES NVIDIA TO BECOME THE MOST PROFITABLE COMPANY IN THE WORLD [Two side-by-side photos: left, a man in a suit (identified by context as a Samsung executive) smiling indoors; right, a man in a suit and glasses (identified by context as Nvidia's Jensen Huang) against a blue-lit backdrop]

@cloneofsimo (Simo Ryu) Reality is "Simplicity is the king" is such normie thing to say. Frontier systems are rarely ever "simple". [Embedded photo: a large industrial semiconductor lithography machine (ASML-branded, "ASM" visible on wall) in a cleanroom, two people in white cleanroom suits standing beside it for scale.] > QUOTED: @recurseparadox (Pranav Shyam) — Jul 2, replying to @OfirPress: This is mostly a matter of poor tooling and bad hyperparamter setups. A complex model can be as much as 10x more effective size if done without confounders. Era of dumb scaling is more over by the ... [platform truncated] 5:08 AM · Jul 3, 2026 · 11.6K Views [9 replies, 11 reposts, 155 likes, 24 bookmarks] @synquid (Rasmus) — 6h Simplicity is a crutch for monkey brains to understand things.

**Simo Ryu** @cloneofsimo [2025-09-09](https://x.com/cloneofsimo/status/1965263486357045567) \* Dont use try-except, like ever \* Dont use cringy emojis, like ever \* make sure to remove any artifacts you generated \* Dont make readme after youve done your job \* Think of the test you would need to pass, write that test, and test your implementation against it. \* Dont fucking celebrate with emojis in front of me if you didnt pass the test you self-generated. \* No, Im not always absolutely right. Im a human, assume you are actually smarter than me \* Dont be a retard and do repeat yourself. I will punch you every time you make an unnessesary class / abstractions. --- **John** @jrysana [2025-09-09](https://x.com/jrysana/status/1965400271233974394) 🤣🤣🤣 --- **kache** @yacineMTB [2025-09-09](https://x.com/yacineMTB/status/1965372665382023173) "i have a script that I run, which will find unnecessary try excepts and unnecessary branches that will throw an error and halt our progress" --- **Joshua Levy** @ojoshe [2025-09-09](https://x.com/ojoshe/status/1965295988094238912) Yep --- **Tushar Reddy** @reddygtvs [2025-09-09](https://x.com/reddygtvs/status/1965340450669388237) when claude code tries to run the dev server for the 1000th time --- **jookiba** @jumbaj00kiba [2025-09-09](https://x.com/jumbaj00kiba/status/1965395795487944811) \* if I see another \`any\` or \`as unknown as\` I will personally unplug you --- **Charles Packer** @charlespacker [2025-09-09](https://x.com/charlespacker/status/1965299309349667087) npm run build hates him! make any ts compile with this one weird trick (“as any”) --- **neonwatty** @neonwatty [2025-09-09](https://x.com/neonwatty/status/1965398644213125512) \*never tell me "🎉 PRODUCTION READY!! 🎉", if you do i will find the server you're running on and smash it to pieces --- **Roman** @worldlive123 [2025-09-09](https://x.com/worldlive123/status/1965421679829811410) Weird because I relate to all of these except the last one. I feel like it never makes any abstractions. You might be scoping it smaller/better --- **Lee Penkman** @LeeLeepenkman [2025-09-09](https://x.com/LeeLeepenkman/status/1965327917099155883) nice... clearly directed at claude haha --- **MantisClone** @MantisClone [2025-09-09](https://x.com/MantisClone/status/1965429447945056427) But I like cringy emojis.

In the very beginning of diffusion training, you can plot gradient std, variance, magnitude, MSE-loss, gradient noise-scale (critical batch size estimation) for different timestep and its beautiful actually [image: six-panel plot grid — Gradient Variance vs Diffusion Timestep, Gradient Std vs Diffusion Timestep, Gradient Noise Scale vs Diffusion Timestep (all showing bell-curve/rising shapes peaking mid-range), Mean Gradient Magnitude vs Diffusion Timestep (sigmoid rise), Average MSE Loss vs Diffusion Timestep (declining sigmoid), and Noise Scale vs Variance colored by timestep (scatter, arc shape)]

This is something I learned naturally that i havent seen anyone else doing, but I always plot log-std / log-change-std plot of each weights once in every T steps. if this is not within 0.01 ~ 1.0 * lr * sqrt(T) slope 'region', I always think im doing something wrong. For example below, i fucked up big time. I wonder if there is name for this. [Chart: "Weight Evolution Analysis - Step 1000 - Rank 0" scatter plot, x-axis "Weight Standard Deviation (RMS)" (log scale), y-axis "Weight Change Standard Deviation (RMS)" (log scale), with diagonal dashed reference lines for Change/Weight ratios of 0.01, 0.1, and 1.0. Bubble size = parameter count, color = log10(parameters). Most layer bubbles cluster near the 0.01 line except one point far above the 1.0 line, flagged as an error.]

I really want to call out examples, but last time I did that I literally got threats based on 'their research networks and future hire rejections' so I decided to not call out names on twitter but to give you couterexamples: Karras in his paper comes up with (I swear to god) > 4 novel methods and just casually name the paper "Analyzing and Improving" Another GOATs, "Simple Diffusion", "Scalable Shampoo", "Improving Diffusion Models", "Diffusion Beats Gans" They just contribute without cringingly renaming and trying to steal previous work's name [Four screenshots of paper title/abstract pages: (1) Karras et al., NVIDIA/Aalto, "Analyzing and Improving..." diffusion paper, 20 Mar 2024; (2) a diffusion paper abstract with example generated images of a frog, an owl playing piano, and robots; (3) Anil, Gupta, Koren, Singer et al. (Google Research/Tel Aviv/Princeton), "Scalable Shampoo" second-order optimization paper, March 8 2021; (4) Nichol & Dhariwal, "Improved Denoising Diffusion Probabilistic Models", 18 Feb 2021]

Simo Ryu @cloneofsimo Many noise from recent 4.5 release, but two important take from vid imo: 1. "GPT4.5 was trained on multiple datacenters" Translate that to "diloco goes brr for largest LLM on the market", bullish on async, low bandwidth training in 2025. 2. "We aggressively used low precision training" -> another use of fp8 training, presumably on h100s. Im guessing they benefited from fp8 because of high granularity 3:44 AM · Feb 28, 2025 · 10.9K Views [4 replies, 4 reposts, 120 likes, 25 bookmarks] Pavel Surmenok @surmenok · 29m It doesn't have to be async. Google is training on multiple datacenters synchronously. You just need a high bandwidth link. [40 views shown] subho ghosh @SubhoGhosh02 · 11h (partially obscured by nav bar) deepseek is way ahead :)