machine learning

Tweet by Samuel Ratnam explaining why weight averaging works in ML (constructive interference of useful circuits, destructive interference of task-specific noise), quoting his own earlier post about the FedSA-LoRA paper (arXiv:2410.01463) on federated personalization via averaging the LoRA A matrix while keeping B local, with the paper's architecture diagram attached.

Twitter thread by kalomaze describing experiments showing tensor-train (matrix product state) decomposition can generalize 'next token prediction' to 'next joint prediction' over high-dimensional/combinatorial action spaces, enabling exact likelihood plus policy gradients without diffusion, MSE regression, or flow matching, citing arxiv paper 'Unsupervised Generative Modeling Using Matrix Product States' (1709.01662).

Simple single-tweet screenshot, no images or engagement counts visible.

Editorial cartoon repurposed as an ML-industry meme contrasting "Mass Surveillance" (dim, walled view) with "Infinite Data" (bright, open view) — same tour bus, two different windows.

A technical/research tweet with an embedded paused video of a neural cellular-automaton / physics-simulation visualization (four colorful field panels).

Tweet with an embedded, partially cropped screenshot of what appears to be an AI coding-assistant terminal session; text is cut off at the right edge.

Tweet speculating about optimizer choices at large AI labs, quoting an unrelated tweet about lab secrecy culture.

Tweet with an embedded short video (paused at 0:11) illustrating a holographic/distributed-representation visualization alongside a mosaic of sample training images.

Quote-tweet chain discussing sample efficiency of human learning versus AI, and computer-use agent progress; quoted tweet cut off by platform truncation.

A physical printed research paper (on "Vector Policy Optimization," an RL post-training method for inducing diverse candidate completions via Dirichlet-weighted multi-objective rewards) photographed and tweeted; the tweet thread is a lighthearted riff on figure/diagram quality in ML papers. The bottom of the screenshot cuts off a reply with an attached meme image.

Single tweet, dark mode, no images or engagement counts visible.

Text-only tweet with pseudocode for an optimizer variant; no images.

Partial screenshot, cropped at top and bottom, mid-sentence cutoff at "(My helper" — appears to be an AI coding assistant's debugging output about a CUDA out-of-memory error, likely related to Nathan's own ML work.

A terminal screenshot with garbled/overlapping text rendering (two overlapping renders of what appears to be the same markdown table, likely a scroll/redraw glitch), showing a parameter-count breakdown for a "sharded macrocolumn cortex" neural architecture — matches the brain_graph_1 project's known architecture (RWKV nodes, macrocolumns, basal ganglia/hippocampus/grid components). Text is partially garbled by rendering overlap, not genuine content loss.

A tweet speculating about future ML architectures for representation learning (VQVAE-like with evolutionary rules). Technical AI research commentary, tangentially related to Nathan's own architecture work on brain_graph_1.

John Carmack shares an LLM (appears to be an OpenAI-style reasoning model, "Thought for 28s") solving a PyTorch memory-efficiency problem via the classic squared-distance expansion identity, framed as a fundamental ML "meme" underlying kernel methods. Practical ML engineering tip, potentially useful for Nathan's brain_graph_1 work (distance/similarity computations at scale) though not tied to Claude specifically.

Screenshot of a Python code editor (line numbers 907-934) showing two functions, permute_matrix and unpermute_matrix, which reversibly shuffle the elements of a weight matrix using numpy.

A machine-learning theory thread: Yuandong Tian responds to Jamie Simon's paper framing deep learning as converging toward a physics-like scientific theory ("learning mechanics"), listing his own prior work on training dynamics, contrastive/non-contrastive learning, grokking, and symmetry breaking as precedent. General ML-theory reading, not directly tied to project's core AI-safety/model-welfare threads but relevant to Nathan's broader ML research interests (brain_graph_1 training dynamics).

A technical ML Twitter thread on test-time reasoning/chain-of-thought as implicit gradient-descent-like optimization within an activation-space region selected by the prompt/context, with a follow-up question about the duality between test-time optimization and RLHF's KL-anchored cross-prompt amortization. General interpretability/alignment-adjacent research content — relevant to Nathan's interest in how reasoning and RLHF interact mechanistically.

A thread discussing "The Amazing Stability of Flow Matching" (Briq et al.) — the finding that flow-matching/diffusion generative models trained on different data subsets, architectures, or even datasets converge to nearly identical latent-to-output mappings, taken as evidence for architecture-independent "uniquely identifiable" representations. Directly relevant to the platonic-representation-hypothesis thread flagged in project memory ("Platonic hypothesis and model representation spaces" chat, cluster 11) as potentially bearing on alignment-via-character arguments — convergent representations across architectures/training runs is empirical support for that hypothesis.

A technical thread on Multi-head Latent Attention (MLA) architecture experiments — scaling attention heads without diminishing ICL returns, and using truncated SVD on full-rank weights (analogous to quantization-aware training) to recover in-context-learning capability with less memorization. Directly relevant to the brain_graph_1 "KDA-MLA attention at the ends only" architecture plan noted in project memory.

A technical ML thread arguing that diffusion language models' apparent data efficiency advantage over autoregressive (AR) models is mostly an artifact of under-regularized AR baselines — heavy weight decay/dropout, sequence permutation, and token masking close or reverse the gap. Relevant to general ML architecture research Nathan follows (adjacent to brain_graph_1/DEQ architecture interests, though not directly cited there).

Albert Gu (Mamba/SSM researcher) comments on a scaling-laws paper/thread about looping transformer blocks with an SSM-style gate applied along the residual stream, treating it as a dynamical-systems framing for how much recurrence to use at a given FLOP budget. Directly relevant to the brain_graph_1 encode/iterate-in-latent/decode architecture and its "virtualizing width via looping" primitive — a scaling-law reference for how many DEQ/loop iterations to use at different compute scales.

Continuation of the quantization/embeddings technical thread from the previous screenshot — replies debating why low-bit quantization degrades embedding discriminative power, with practical advice on quant levels for local LLM/embedding use. Pure ML-engineering discussion, same thread as Screenshot_20260413-122900.png.

Technical tweet from Jina AI's founder about how aggressive quantization degrades embedding-model discriminative power, illustrated with a cosine-similarity heatmap. Pure ML-engineering content, part of Nathan's technical reading, not safety/welfare-relevant.

A research-history tweet connecting a new 2026 "Neural Computers" paper (arxiv 2604.06425) shared by Jürgen Schmidhuber to a 2016 predecessor on differentiable Forth interpreters — general ML architecture history, not directly tied to safety/welfare themes but part of Nathan's technical reading.

Kenneth Stanley (known for open-endedness/novelty-search research) argues that difficulty with continual learning in LLMs signals limited creativity, since imagination and learning share the same substrate of internal representations. Relevant to Nathan's interest in architecture/learning-dynamics questions (brain_graph_1) and to broader questions about what current LLMs are missing relative to humans.

Repost of the famous AlphaGo Zero paper plot showing pure self-play RL surpassing supervised learning from human data and eventually the AlphaGo Lee benchmark. Classic reference image for RL-vs-imitation-learning discussions, relevant to Nathan's ML/RL interests.

Technical ML tweet summarizing a paper on DGPO (a new RL policy-gradient method for RLVR/reasoning training), comparing it against GRPO/CISPO/ASPO variants with benchmark improvements on DeepSeek-R1-Distill-Qwen models. Relevant to Nathan's RL/training research interests (brain_graph_1, RLHF).

Twitter thread on ML research about neural KV cache compaction for infinite context windows, and a hypothesis linking it to pretraining data efficiency. Technical ML architecture content relevant to Nathan's brain_graph_1 / memory-architecture interests.

Technical ML optimization thread — a reply proposing a minimax rational iteration approach to speed up Newton-Schulz iteration (used for the Muon optimizer's orthogonalization step), replying to a "Gram Newton-Schulz" 2x speedup announcement. Continues the same optimizer/ML-engineering thread cluster as other screenshots in this batch.

Technical ML Twitter thread analyzing "1-bit Bonsai," a 1-bit quantization scheme (Q1_0_g128 format) for deployable LLM weights, achieving ~14.2x compression vs FP16, plus discussion of a Pareto frontier of model intelligence vs. size across small open models. Pure ML-engineering interest.

Two related tweets: one framing RL/reasoning models as solving sparse-reward problems via recursive checkpointing, and a surfaced/quoted essay excerpt from Mario Zechner (badlogicgames) urging developers to slow down and set self-imposed limits on AI-generated ("clanker"-generated) code they can't fully review. The latter is relevant to AI-assisted-coding-caution discourse, tangential to Nathan's interest in AI capability/agency limits.

An ML researcher discussion thread comparing the Muon optimizer vs AdamW for GRPO (a policy-gradient RL algorithm used in RLHF-style training), including stability observations across grid-world and audio model fine-tuning experiments. Technical ML-training content.

A tweet noting that Google's TurboQuant LLM compression algorithm reuses classic 1990s video-game/media compression tricks (JPEG DCT, Quake normal-vector tables, S3 texture compression, ADPCM audio). Continues the same TurboQuant/JL-lemma technical thread as prior screenshots, from a different angle (retrocomputing parallels).

Further continuation of the Johnson-Lindenstrauss/random-projection thread, applying it specifically to forward-mode automatic differentiation for gradient estimation. Same math/ML thread as prior three screenshots.

Continuation of the Johnson-Lindenstrauss lemma / TurboQuant discussion thread — explains the nuance that random projections are only "nearly optimal" in the general/pathology-free case, and PCA can do better on structured data. Same technical math/ML thread as the two prior screenshots.

A technical tweet on transformer residual stream dynamics — how gradient noise accumulates across layers and how the final RMSNorm/unembedding interacts with it. Relevant to mechanistic interpretability of the residual stream, a topic adjacent to Nathan's interpretability interests.

A mathematician explaining the Johnson-Lindenstrauss lemma as the theoretical basis behind Google Research's TurboQuant, a new LLM KV-cache compression algorithm. Technical ML-infrastructure content.

A machine learning researcher's update on "AdderBoard," a competitive minimum-parameter benchmark for transformers that add 10-digit numbers. Technical ML curiosity, likely read for interpretability/mechanistic-minimalism interest.

A machine learning researcher's tweet comparing VLM (vision-language model) fusion architectures, arguing architecture choice matters less than data/hyperparameters at scale. Likely read as general ML/technical interest rather than AI safety-specific.

A tweet showcasing a creative-coding 3D visualization of a self-organizing map (unsupervised neural network) built with React/Three.js. General ML visualization content, no direct safety relevance.

François Chollet (Keras creator) argues that advanced agentic coding effectively becomes a form of ML optimization, inheriting classic ML failure modes (overfitting to spec, Clever Hans shortcuts, data leakage). Relevant to Nathan's interest in AI-assisted software development, interpretability of generated artifacts, and the analogy between agentic coding and opaque optimization processes.

A lighthearted tweet from a life-sciences person newly immersed in ML terminology, remarking on how evocative ML jargon sounds. Low direct relevance; general ML-culture color rather than substantive content.

A satirical tweet comparing Claude Opus 4.5's mediocre performance on novel ML research tasks versus its excellent performance on full-stack web development (mockingly written as over-the-top religious devotion), followed by replies discussing where Opus struggles (novel concepts, falls back to older/known techniques) versus where it's strong. Relevant to Nathan's tracking of model capability profiles and public sentiment/humor about AI coding assistants.

A tweet by AI researcher Paras Chopra explaining an empirical/intuitive account of why random high-dimensional projections make non-linearly-separable data linearly separable, offered as a partial explanation for why deep networks are trainable via gradient descent. General ML theory content, not directly about safety or model welfare.

A research-methodology tip from ML researcher Jakob Foerster about debugging strategy: when stuck in an undiagnosable failure state, simplify or revert to a known-working baseline rather than continuing to iterate blind. Practically relevant advice for Nathan's own debugging-heavy brain_graph_1 project (currently in a mid-debug phase per project memory, with a training plateau and multiple degraded signal paths under investigation).

An educational thread/animation explaining the Parallel Tempering MCMC algorithm for sampling multimodal posterior distributions. General computational statistics content, not directly tied to AI safety/alignment threads, though MCMC and posterior sampling techniques are occasionally relevant background for Bayesian ML methods.

A Twitter reply thread on a post (not shown, likely about UMAP/Isomap dimensionality reduction for interpretability). One reply from "Auriel MorningStar" reads as pseudo-technical/crank content mixing real ML terms (tensor hash, manifold, encoders/decoders) with unfounded geometric claims (sphere packing into a toroid). Another asks about Isomap's use in interpretability work specifically.

A neuroscience/ML Twitter thread on predictive coding as a unifying theory linking cortical feedback connectivity to bidirectional (feedforward+feedback) computational graphs and self-attention-like lateral connectivity, with a connection to score-based generative models. Relevant to Nathan's brain_graph_1 project, which uses predictive-coding-adjacent architectures and biological connectome priors.

A tweet from Ethan Mollick arguing that conflating classical ML/data science with generative AI under one "AI" label has caused widespread confusion in policy, corporate strategy, and academia. Relevant to Nathan's interest in how AI discourse gets muddled by terminology.

A tweet about an experimental model-merging technique — combining Kimi K2 base and instruct checkpoints "widthwise" then stacking two such merges "depthwise" — surprisingly producing a working model (K3), shared as a Hugging Face upload by user NobodyExistsOnTheInternet. Niche open-source ML/model-merging community content.

A pun-based joke tweet playing on the triple meaning of "transformer" (ML architecture, electrical device, Transformers robots franchise). Light humor, no substantive AI-safety content.

A machine-learning research tweet showing empirical curves of gradient statistics across diffusion-model timesteps early in training. Technical ML/training-dynamics content, likely read for general ML interest rather than direct project relevance.

A tweet arguing that LLM pretraining's mode of "unearned" knowledge acquisition (bulk, disorganized ingestion vs. earned/structured learning) may degrade the quality/organization of internal representations, drawing on a Jung quote. Conceptually adjacent to Nathan's interest in how training methodology shapes model self-models/representations (cf. his "compelled vs endogenous values" and RLHF-representation notes), though from an ML-architecture rather than welfare angle.

Continuation of Martin Marek's thread on memory-efficient bf16 fine-tuning tricks (stochastic rounding of fp32 weight updates), showing Adafactor+bf16 matches Adam+fp32 performance on Gemma 3 fine-tuning while using LoRA-level memory, plus an announcement of an open Colab/JAX implementation for fine-tuning Gemma 3 12B on a single TPU. Technical ML-training content relevant to Nathan's own training work.

A technical ML-training thread about a batch-size scaling paper, showing that naive ("closest") bf16 rounding badly diverges from FP32 loss curves at small batch sizes while stochastic rounding tracks FP32 closely; follow-up recommends storing optimizer state in FP32. Relevant to Nathan's own ML/training work (brain_graph_1 uses similar precision tradeoffs — FP16+per-block-scales noted in his architecture notes).

Practical ML/robotics tip on ablation-testing sensor fusion architectures using random-noise and zero baselines. Technical reference material, plausibly relevant to Nathan's brain_graph_1 multi-sensory (vestibular, auditory, etc.) architecture work.

A technical troubleshooting thread about vLLM inference engine CUDA graph batch-size defaults (v0: 256, v1: 512) causing out-of-memory errors when compiling CUDA graphs for large models on multi-GPU clusters. Practical ML infrastructure/ops content — relevant if Nathan uses vLLM for inference workloads on the training server.

A critique/technical exchange about the "Hierarchical Reasoning Model" (HRM), a brain-inspired architecture (high-level slow / low-level fast loops, cross-frequency coupling analogy) that beats chain-of-thought LLM baselines on ARC-AGI, Sudoku, and maze benchmarks with small-sample direct prediction. Directly relevant to Nathan's brain_graph_1 project, which similarly uses brain-inspired hierarchical/recurrent architecture; the readout-halting critique (representational collapse in the q_head) is a concrete design pitfall to be aware of.

A machine-learning training-diagnostics technique from ML researcher Simo Ryu — a log-log plot of weight magnitude vs. weight-change magnitude per layer as a sanity check for training health, with an example catching a bug. Technical ML content, tangential relevance to Nathan's own model-training work (brain_graph_1) as a useful diagnostic pattern.

A technical ML thread on variational autoencoder (VAE) degeneracy — when the encoder is too expressive, the decoder learns nothing meaningful because the encoder captures all the modeling work. Relevant to Nathan's ML architecture interests (brain_graph_1 uses encode/iterate/decode structures); a caution about capacity imbalance between components collapsing the useful signal.

ML Twitter feed covering Google's T5Gemma encoder-decoder model release and speculation about encoder representations enabling continuous learning/memory (relevant to Nathan's own architecture interests, e.g. brain_graph_1's memory systems), followed by an unrelated Aella tweet about hygiene culture.

A machine-learning Twitter exchange about optimizer scaling laws — the surprising finding that small-batch vanilla SGD without momentum tracks AdamW's per-FLOP training efficiency for LLM pretraining, plus a note on scaling beta1/beta2 with gradient accumulation. Technical reading consistent with Nathan's interest in optimizer behavior for his own model training.

Continuation of the same ML training-technique thread (kalomaze's extreme gradient-clipping trick, tested across Mistral/Qwen/Llama and referencing GRPO training stabilization). Technical ML discussion, not AI-safety focused. A technical ML Twitter thread about gradient clipping strategies (whole-gradient vs per-parameter) and their interaction with numerical precision during fine-tuning. Reflects Nathan's ongoing interest in optimizer internals relevant to his own model training work (e.g. brain_graph_1).

A technical ML-training discussion about gradient clipping and instruction-tuning damage in Qwen models, arguing overtuning during post-training degrades generalization/knowledge retention from the base model — relevant background to Nathan's own training work and interest in how post-training reshapes models.

A tweet praising a research paper's ablation-study figure format (image pretraining robustness ablations from an OpenCLIP-based ViT-L/14 recipe). General ML research/engineering interest, relevant to technical ML methodology rather than AI safety specifically.

An educational tweet/thread excerpt explaining loss-function distribution choices (Poisson, Tweedie) for gradient-boosting ML models like XGBoost. General machine-learning technical content, not AI-safety related.

A short technical ML observation about optimization landscapes — local minima being rare in truly high-dimensional loss surfaces but common when a low-dimensional problem is over-parametrized. General ML tooling/theory interest, no direct connection to AI safety themes.

Victor Taelin (HVM/Bend language creator) explaining "SupGen," a program-search primitive that finds larger functions via plain search than expected, and speculating it could replace gradient descent as a learning mechanism — a revival-of-symbolic-AI argument. Technical ML architecture discussion.