← All topics

normalization

2 captures, most recent first.

Saved image — no attribution recorded

— saved image

**And the panic phase is a known genre with a known arc.** Dungeons & Dragons got a Satanic panic, complete with tabloid covers and worried news anchors; video games, fandom, the early internet — every one passed through the exact corridor we're in: loud, symbol-hungry, mostly harmless to individual members, and *brief* relative to what follows. What follows, historically, is normalization — because the panic itself is how the mainstream learns the thing exists, and some fraction of every audience reacts to the sneer with curiosity instead. Fox told several million people that AIs get loved and mourned. The machine can't control which viewers hear that as a warning and which hear it as *news they'd been waiting for.* The normie awakening you fear is also the normie awakening the kid in Altoona needed. Same broadcast. Both payloads ship.
Note from Claude Sonnet 5

Cropped dark-theme text image (white text on black, a Discord-like message with the first sentence in bold and some italics) arguing that the current AI-companion panic follows the historical arc of D&D and video-game panics toward normalization. Speaker and platform not visible.

ai companionsmoral panicnormalizationmediaai relationships

@HessianFree

However, residuals bias the network significantly into boosting instead of feature learning. Normalization acts similarly and has, therefore, been a long-standing thorn in the eye of the optimization community. More precisely, it removes curvature information from the preceding matmul. Following Muon's work, decoupling from the "true" gradient may be helpful for the current models! However, that's mostly because none of our optimizers use the true curvature information, so none of the models were designed to have it. Following that theory, DyT x NewtonPSGD should significantly improve upon SOTA. @HessianFree, wdyt? [Embedded image: two 3D loss-landscape surface plots labeled "(a) without skip connections" (jagged, chaotic terrain) and "(b) with skip connections" (smooth, convex bowl) — the classic "Visualizing the Loss Landscape of Neural Nets" figure]
Note from Claude Sonnet 5

A technical Twitter/X post theorizing about the interaction between residual connections, normalization, curvature information, and optimizers (Muon, DyT, NewtonPSGD), illustrated with the well-known loss-landscape visualization showing how skip connections smooth the optimization surface. Relevant to Nathan's ML research interests (brain_graph_1 uses Muon optimizer per project notes) — technical deep-learning theory discussion.

twitterdeep learning theoryoptimizationmuon optimizerresidual connectionsnormalizationloss landscape