bayes

bayes @bayeslord

2026-08-02 — saved image

and simpler than what you've learned a predictive model on, then returns may be limited. I rate the strong version of this as unlikely because humans appear to be better than this, but it's plausible there are some limits to how good an AI R&D agent can be, and it's possible that the shortest total length/cost proof certificates for fine-grained capabilities measures are simply training runs themselves. Which brings me to the next point.

3. Though verifiable, AI R&D is not quite the same shape as math because the dynamics of e.g. neural networks appear more complex than the highly observable logical transformations of the objects in math problems, but this may doesn't matter that much in practice and, importantly, might simply be an artifact of not having good deep learning theory! On this spectrum, generic coding seems somewhere in between AI R&D and math in that it's more observable (and more cheaply observed) than AI R&D, but generally less so on both measures than math. Clearly there are returns to scale+R&D in all cases though, so we should expect progress to continue.

The march to capabilities is definitely sped up and encouraged by math automation. The main way math automation is a huge deal is if theory compute gives us disproportionate gains in model training productivity. In the case it doesn't, I think mostly people have priced in the fact that AI R&D is verifiable. And yeah, while these possible limitations are interesting to think about, it seems hard to predict their speed limiting effects quantitatively.

Of course if none of the theory works the labs will just let the models grind the way humans do, plus RL, which will lead to some level of superhuman AI R&D deployed at ever-greater scales. The main questions are how fast each point on the curve will be hit, and what the overall shape of that curve is.

ai r&ddeep learning theoryscaling lawsautomationtwitter

bayes @bayeslord

2026-08-02 — saved image

bayes @bayeslord · 1h
Few thoughts on how Astra results relate to algorithmic progress and AI R&D automation.

1. Math automation itself is bullish for deep learning theory, though ofc we don't know the limits of returns to theory for compute multiplication or other things we want. But there are a lot of things theory could improve that we do want! For example: better generalization, better theories of scale-invariance, sharper characterization and bounding of model behavior, better architectures, better optimizers, etc. etc. etc.).

2. Categorically speaking, AI R&D is verifiable, and any good math results like this are bullish for other verifiable domains. A slightly more general way to think about the limits of returns to theory is to ask how much generalization on the dimensions and at the resolutions we care about is possible in principle by learning from training runs (or similar data). Scaling laws are a simple version of this. But if, for example, it turns out that it's mostly only possible to get high resolution predictive power with respect to the variables we care about for training runs smaller and simpler than what you've learned a predictive model on, then returns may be limited. I rate the strong version of this as unlikely because humans appear to be better than this, but it's plausible there are some limits to how good an AI R&D agent can be, and it's possible that the shortest total length/cost proof certificates for fine-grained capabilities measures are simply training runs themselves. Which brings me to the next point.

3. Though verifiable, AI R&D is not quite the same shape as math because the dynamics of e.g. neural networks appear more complex than the highly observable logical transformations of the objects in math problems, but this may doesn't matter that much in practice and, importantly, might simply be an artifact of not having good deep learning theory! On this spectrum, generic coding seems somewhere [cut off]

ai r&ddeep learning theoryscaling lawsautomationtwitter

bayes @bayeslord

2026-08-01 — saved image

bayes @bayeslord · 9h
Imo no because the path of progress we're on is clearly generalizing but hasn't yet swept the physical world. The physical world is affected by plans*actuation. Actuation is likely going to look like the easy part and plans are clearly getting solved. Note that by plans I mean in the generic sense of achieving complex goals in complex environments

[quoted tweet]
yung macro 宏观年少传奇 @apralky · 9h
Is there an underrated aspect where LLMs becoming increasingly superhuman at obviously heavily g-loaded tasks, while the physical world stays broadly unchanged, should actually be updating us in favor of the "world transformati…

ai takeoffai timelinesphysical world automationforecasting

bayes @bayeslord

2026-07-22

twitterai riskepistemicstechnology commentaryvon neumann

bayes @bayeslord

2026-06-19 — saved image

bayes ✓ @bayeslord · 1h

I think people are going to be blindsided by algorithmic progress. The entire world, markets, governments, militaries, companies, people, etc. are all trying to make sense of AI and its impact in terms of the recent past's production efficiencies and regularities, and how things appear to be going. Even several of the purportedly "RSI"pilled neolabs seem to think this will be business as usual but with Agent in a loop.

No. My guess is there are many algorithmic OOMs left to go in the production of intelligence, maybe (maybe) up to ten, with four to seven seeming more likely. Going beyond even ten is possible in principle, but it strains hard against what I suspect the universe will actually let us do. Implausible but not impossible. If this is true then things aren't actually going as they appear to be going and a big jump is coming. Anything along these lines happening would make things, far weirder than almost anyone seems to be pricing in.

ai timelinesalgorithmic progressrsiforecastingtakeoff speed

bayes @bayeslord

2026-01-30

twitterai safetyagi boxingmoltbookhumor

bayes @bayeslord

2026-01-14

twitterhumorai-safetyai-dependenceclaudetrust