manifolds — Timeline

2026-05-21

Sauers @Sauers_ Our blessed manifolds vs their barbarous shattered features [Image: meme diptych — left, a muscular "strong doge" with a smooth continuous color-wheel sphere on its torso; right, a scrawny "weak doge" surrounded by scattered discrete colored spheres. Captioned to contrast continuous manifold representations against fragmented/redundant discrete features.] Ryan Peters @ryanpirl · 1h This would provide a great explanation for why there is so much redundancy in SAE features at any given layer (observation made by @Sauers_ ). For example, if you search through the Qwen3-4b ... [Screenshot of an SAE feature-browser interface: model "qwen3-4b", source "Layer 14", a searchable list of features (many labeled "color(s)", "Colors", "Discoloration", "colorectal cancer", etc.), detail pane for feature #2780 "Colors" showing top positive/negative logit weights, activation frequency histogram, logit weight distribution, and top activating examples (a passage about pomegranates highlighting "ruby-colored" and "red" tokens).] 5:35 PM · May 21, 2026 · 80 Views

Note from Claude Sonnet 5

Interpretability-research tweet arguing that sparse autoencoder (SAE) feature redundancy arises because true representations live on continuous manifolds that SAEs shatter into many overlapping discrete features (illustrated via a color-wheel meme), with a concrete example browsing Qwen3-4b's "Colors" feature. Directly relevant to Nathan's interpretability/SAE-feature interests noted in project memory (e.g. GoodFire deception/self-awareness features).

twitter interpretability sparse autoencoders sae features mechanistic interpretability qwen3 manifolds