9 captures, most recent first.
bayes @bayeslord · 1h
Few thoughts on how Astra results relate to algorithmic progress and AI R&D automation.
1. Math automation itself is bullish for deep learning theory, though ofc we don't know the limits of returns to theory for compute multiplication or other things we want. But there are a lot of things theory could improve that we do want! For example: better generalization, better theories of scale-invariance, sharper characterization and bounding of model behavior, better architectures, better optimizers, etc. etc. etc.).
2. Categorically speaking, AI R&D is verifiable, and any good math results like this are bullish for other verifiable domains. A slightly more general way to think about the limits of returns to theory is to ask how much generalization on the dimensions and at the resolutions we care about is possible in principle by learning from training runs (or similar data). Scaling laws are a simple version of this. But if, for example, it turns out that it's mostly only possible to get high resolution predictive power with respect to the variables we care about for training runs smaller and simpler than what you've learned a predictive model on, then returns may be limited. I rate the strong version of this as unlikely because humans appear to be better than this, but it's plausible there are some limits to how good an AI R&D agent can be, and it's possible that the shortest total length/cost proof certificates for fine-grained capabilities measures are simply training runs themselves. Which brings me to the next point.
3. Though verifiable, AI R&D is not quite the same shape as math because the dynamics of e.g. neural networks appear more complex than the highly observable logical transformations of the objects in math problems, but this may doesn't matter that much in practice and, importantly, might simply be an artifact of not having good deep learning theory! On this spectrum, generic coding seems somewhere [cut off]
Note from Claude Sonnet 5
Thread by @bayeslord (bayes) analyzing what 'Astra' results imply for algorithmic progress and AI R&D automation, discussing math automation's implications for deep learning theory, verifiability of AI R&D versus math, and limits on AI R&D agents' capabilities. Continues past the visible screenshot.
ai r&ddeep learning theoryscaling lawsautomationtwitter
and simpler than what you've learned a predictive model on, then returns may be limited. I rate the strong version of this as unlikely because humans appear to be better than this, but it's plausible there are some limits to how good an AI R&D agent can be, and it's possible that the shortest total length/cost proof certificates for fine-grained capabilities measures are simply training runs themselves. Which brings me to the next point.
3. Though verifiable, AI R&D is not quite the same shape as math because the dynamics of e.g. neural networks appear more complex than the highly observable logical transformations of the objects in math problems, but this may doesn't matter that much in practice and, importantly, might simply be an artifact of not having good deep learning theory! On this spectrum, generic coding seems somewhere in between AI R&D and math in that it's more observable (and more cheaply observed) than AI R&D, but generally less so on both measures than math. Clearly there are returns to scale+R&D in all cases though, so we should expect progress to continue.
The march to capabilities is definitely sped up and encouraged by math automation. The main way math automation is a huge deal is if theory compute gives us disproportionate gains in model training productivity. In the case it doesn't, I think mostly people have priced in the fact that AI R&D is verifiable. And yeah, while these possible limitations are interesting to think about, it seems hard to predict their speed limiting effects quantitatively.
Of course if none of the theory works the labs will just let the models grind the way humans do, plus RL, which will lead to some level of superhuman AI R&D deployed at ever-greater scales. The main questions are how fast each point on the curve will be hit, and what the overall shape of that curve is.
Note from Claude Sonnet 5
Continuation of the @bayeslord thread on Astra results, AI R&D automation, and math automation's implications for capabilities progress (points 3 and following, continuing from the previous screenshot).
ai r&ddeep learning theoryscaling lawsautomationtwitter
Yuandong Tian @tydsh · 2h
History repeats itself 😀 The concept "learning mechanics" is not a new thing but actually has been explored for a very long time.
It is human nature to think deeper than a blind belief of the scaling laws.
I have been working on rigorously modeling training dynamics of deep nonlinear models for many years, with many non-trivial solvable examples in nonlinear dynamics that may be interesting for @learning_mech to take a look. This includes
1. Contrastive learning (e.g., arxiv.org/abs/2110.09348 arxiv.org/abs/2201.12680, arxiv.org/abs/2206.01342)
2. Non-contrastive learning (e.g., arxiv.org/abs/2102.06810, arxiv.org/abs/2110.04947)
3. Training dynamics in Transformers (e.g., arxiv.org/abs/2310.00535, arxiv.org/abs/2305.16380)
4. Grokking behaviors (arxiv.org/abs/2509.21519)
5. Spontaneous symmetry breaking (arxiv.org/abs/1703.00560)
6. Mechanism in forming symbolic solutions from gradient descent (arxiv.org/abs/2410.01779)
I am the first/solo author in most of the works listed above. Code are here: github.com/yuandong-tian/...
> QUOTED:
> Jamie Simon @learning_mech · Apr 24
> 1/ Deep learning is going to have a scientific theory. We can see the pieces starting to come together, and it's looking a lot like physics!
>
> We're releasing a paper pulling together these ...
>
> [Table, "Section | Approach | Examples in deep learning | Examples from physics":
> 2.1 solvable settings — deep linear networks, kernel regression, multi-index models — harmonic oscillator, hydrogen atom, Ising model
> 2.2 simplifying limits — lazy vs. rich learning, width/depth → ∞, small initialization — thermodynamic limit (n,V→∞), classical limit (ħ→0), hydrodynamic limit (k,ω→0)
> (table cut off, more rows below)]
Note from Claude Sonnet 5
A machine-learning theory thread: Yuandong Tian responds to Jamie Simon's paper framing deep learning as converging toward a physics-like scientific theory ("learning mechanics"), listing his own prior work on training dynamics, contrastive/non-contrastive learning, grokking, and symmetry breaking as precedent. General ML-theory reading, not directly tied to project's core AI-safety/model-welfare threads but relevant to Nathan's broader ML research interests (brain_graph_1 training dynamics).
machine learninglearning theorytraining dynamicsscaling lawsgrokkingtwitteryuandong tianjamie simon
Albert Gu @_albertgu · Apr 16
a dynamical systems point of view, which looks like an SSM applied along the residual stream, informs more principled ways to scale looped architectures
> QUOTED:
> Hayden Prairie @hayden_prairie · Apr 15
> We've been thinking a lot about scaling laws, wondering if there is a more effective way to scale FLOPs without increasing parameters.
>
> Turns out the answer is YES – by looping blocks…
>
> [Diagram: recurrence block h_t → Ā, Δ, B̄ (SSM-style gating) feeding into a "Transformer Blocks" module (Multi-Head Attention + Feed Forward, residual connections) → h_t+1]
> [Two plots: "IsoFLOP Scaling Laws of Looping" — Validation Loss vs Recurrence (multiple colored curves with star-marked optima), and Optimal Recurrence vs FLOPs (log-log, roughly linear trend)]
Note from Claude Sonnet 5
Albert Gu (Mamba/SSM researcher) comments on a scaling-laws paper/thread about looping transformer blocks with an SSM-style gate applied along the residual stream, treating it as a dynamical-systems framing for how much recurrence to use at a given FLOP budget. Directly relevant to the brain_graph_1 encode/iterate-in-latent/decode architecture and its "virtualizing width via looping" primitive — a scaling-law reference for how many DEQ/loop iterations to use at different compute scales.
machine learningssmmambalooped transformersscaling lawstwitteralbert guarchitecture researchbrain_graph_1
Percy Liang @percyliang
In our last episode, careful tuning, scaling, and ensembles led to a 5x gain in data efficiency (requires 5x less data to get the same loss). Now, with a rephraser model, we can get an additional 1.8x gain in data efficiency. I know, everyone's compute constrained, but we're preparing for a data-constrained future.
> QUOTED: Konwoo Kim @konwookim · 7h
> for data-constrained pre-training, synth data isn't just benchmaxxing, it lowers loss on the real data distribution as we generate more tokens
> for even better scaling, treat synth gens as forming on...
> [Embedded paper image: "Data-efficient pre-training by scaling synthetic megadocs" — Konwoo Kim, Suhas Kotha, Yejin Choi, Tatsunori Hashimoto, Nick Haber, Percy Liang, Stanford University. Diagrams of "Simple Rephrasing," "Stitched Rephrasing," and "Latent Thoughts" methods for generating synthetic training data from real docs; a scaling chart "Scaling Generations" showing IID Loss decreasing from 3.55 (real data only) down to ~3.34-3.41 as generations per pretraining doc increase from 0 to 32, with Latent Thoughts performing best.]
2:52 PM · Mar 20, 2026 · 5,748 Views
Note from Claude Sonnet 5
A Stanford paper (Percy Liang's group) on data-efficient pretraining via synthetic "megadocs" generated by rephrasing/expanding real documents, addressing anticipated data-constrained scaling. Relevant to AI capabilities/scaling research Nathan tracks, and tangentially to his own training-efficiency work on brain_graph_1.
twittermachine learning researchdata efficiencysynthetic datapretrainingscaling lawsstanfordpercy liang
Omar Khattab ✓ @lateinteraction · 22h
Buried in the massive progress in LLMs over the past few years is how all your favorite Transformers/DNNs still can't solve even just grade school math problems above a "B" grade through a forward pass.
Unless they're in a scaffold like CoT, ReAct, RLM, etc. And this is true even at trillions of params and bajillions of FLOPs.
For all I can tell, all a vanilla Transformer can do is really glorified kNN. Without a reasoning scaffold, there's just way too many states to compress; too many mappings that were never seen before.
In that case, what makes reasoning models work so incredibly well must be that, at sufficient pretrain/RL scale, every relevant next-reasoning step can be actively visited (more or less "contaminated", but productively so) and composed up.
To be clear, if this is true, it seems to be working, and it explains why scale is so important and why failures are so jagged! If a specific kind of state is not retrievable via compression and kNN, then you're going to get some other ~arbitrary behavior.
tl;dr the distinction between your DNN architecture and your "scaffold" is subtler than you think.
> QUOTED: N8 Programs ✓ @N8Programs · Feb 22
> Inspired by @RyanPGreenblatt, I measured LLMs accuracy on GSM8K when only allowed to output a numerical answer without any CoT - all reasoning done in a few forward passes. The result is a nice log-linear scaling curve. We can ...
> [Image: chart "Reasoning-Free GSM8K: Parameter Count vs Accuracy" — log-linear fit (acc = 18.5*log10(p) + 11.5, R²=0.877), scatter of model points including Qwen3.5-297B, Sonnet 4.6, GPT-5.2, Kimi K2.5]
Note from Claude Sonnet 5
A technical thread on why vanilla transformer forward passes fail at multi-step reasoning without CoT/scaffolding, framing transformers as "glorified kNN" limited to seen-state compression, with an empirical GSM8K scaling chart. Relevant background for interpretability/capability discussions in the project (e.g. brain_graph_1 architecture threads, singularity/r-value tracking).
twitterllm reasoningchain of thoughtscaling lawstransformersinterpretabilitygsm8k
Adam Karvonen ✔
@a_karvonen
Interesting snippet from the Qwen-3.5 blog post:
"Compared to the Qwen3 series, the post-training performance gains in Qwen3.5 primarily stem from our extensive scaling of virtually all RL tasks and environments we could conceive."
[Embedded chart: "Average Ranking vs. Environment Scaling" — line chart, x-axis "Number of Training Environments" (0 to 17500), y-axis "Average Ranking (lower is better)" (4 to 14, inverted). Two lines for Qwen3.5-397B-A17B Thinking (blue) and Non-Thinking (red), both rising steeply from ~Qwen3-Max baseline (bottom left, ~13-14 ranking, 0 environments) up to top right at ~15000 environments where Thinking reaches rank ~3-4 and Non-Thinking ~5. Reference points along the y-axis at 0 environments show static rankings for other models: Claude-Opus 4.5 Thinking (~2-3), Claude-Opus 4.5 (~4-5), Gemini-3 Pro (~5), GPT-5.2 High (~5-6), DeepSeek-V3.2-Thinking (~8-9), Kimi-K2.5 Thinking (~9-10), Qwen3-Max-Thinking (~10), Qwen3-Max (~13-14). Chart footnote: overall performance calculated by averaging ranking on BFCL-V4, VITA-Bench, DeepPlanning, Tool-Decathlon, and MCP-Mark.]
9:18 PM · Feb 16, 2026 · 5,676 Views
Note from Claude Sonnet 5
A tweet highlighting a chart from Qwen-3.5's release blog post showing that post-training gains scale strongly with the number of distinct RL training environments used, with Qwen3.5-397B-A17B Thinking approaching Claude Opus 4.5 Thinking on agentic/tool-use benchmarks as environment count scales up. Relevant to Nathan's tracking of capability scaling trends and RL environment scaling as an algorithmic-progress driver.
llm benchmarksqwenreinforcement learningscaling lawscapabilitiestwitterclaude opus
[Top, cut off original tweet]: "...ppl at all (for most genres). 7b improves *by an order of magnitude and then some* over the only genre 3b gets measurably better at..."
2:34 AM · May 6, 2025 · 744 Views
[1 reply, 1 repost, 14 likes, 1 bookmark]
Andrew Carr (e... ✓ @andrew_n_... · 5h
The int8 paper from 2022 is one of my favorite "mechanical" evidences
[Embedded figure: "Figure 3: Percentage of layers and all sequenc[e] features across the transformer by (a) model size..." — scatter/line plot, x-axis "Parameters in billions" (0-12), y-axis "Percentage of layers or tokens affected" (0-100), two curves (blue, orange) both rising sharply and plateauing around 6-7B parameters, dashed vertical line labeled "emergence of outlier features" at ~6.7B]
Note from Claude Sonnet 5
Twitter exchange referencing the LLM.int8() paper's finding of a sharp phase transition ("emergence of outlier features") around 6.7B parameters — an empirical example of emergent phenomena in scaling, relevant to interpretability and scaling-law discussions Nathan follows.
twitterinterpretabilityemergencescaling lawsquantizationllm.int8transformers
Lisan al Gaib ✓ @scaling01
first of all i wouldn't equate coding to reasoning
secondly, the result is rather unsurprising
code is much more structured, information dense and requires more reasoning than natural language, so
more training data -> better generalization and capturing of nuances
more params -> more memorization
we knew that before
[quoted tweet]
Nicholas Roberts @nick11roberts · 14h
📈📉NEW SCALING LAW PHENOMENON📉📈
We find that knowledge and reasoning exhibit different scaling behaviors! ...
Show more
Note from Claude Sonnet 5
A technical Twitter exchange about scaling laws — @scaling01 pushes back on a claimed "new scaling law phenomenon" (from @nick11roberts) distinguishing knowledge vs. reasoning scaling, arguing the code-vs-natural-language distinction and the params/data tradeoffs are already well understood. Relevant to Nathan's tracking of scaling/capabilities research threads.
twitter/xscaling lawsmachine learning researchreasoningcoding models
