Nathan 🔍 reposted
Peter Wildeford... @peterwildef... · 9h
AI timelines -
I've been souring lately on the idea of predicting an arrival date for 'superintelligence' and 'recursive self-improvement' milestones, because this implies that everything prior to this date will be relatively chill and normal, and I don't think that's the case.
But if you define 'runaway recursive self-improvement is possible' as a situation in which AIs can replace highly skilled expert human labor in all aspects of the AI research and development process ('superhuman AI researcher' in the AI2040 framework or 'AI research supremacy' in Cotra's framework). I think it is 50-50 we will reach this milestone in 4 years or earlier.
My 80% confidence interval for this date of runaway RSI is 1-30 years, as there is a long tail where capability progress plateaus.
This also means there is a ~10% chance that we are faced with the possibility of runaway RSI in less than a year's time, similar to what AI2027 predicts.
Note from Claude Sonnet 5
Tweet by Peter Wildeford (reposted by an account named 'Nathan') giving his probabilistic forecast for when AI could achieve 'runaway recursive self-improvement,' defined as replacing expert human AI researchers, with a median of 4 years and an 80% CI of 1-30 years.
Jaime Sevilla [verified] @Jsevillamol · 3h
Why is human-level AI R&D ability a privileged milestone? Why not expect that fooming requires significantly more than human-level AI R&D capabilities? Or significantly less?
[8 replies, 1 repost, 23 likes, 1.7K views]
Ryan Greenblatt [verified] @RyanGreenblatt · 2h
Currently, human labor for R&D and AI labor are complements rather than substitutions. Thus, to reach very extreme rates of progress you probably need more capable AIs such that you're in the substitute regime. These AIs may still be significantly worse than humans in some ways.
Note from Claude Sonnet 5
Twitter exchange between Jaime Sevilla and Ryan Greenblatt debating whether human-level AI R&D ability is a meaningful milestone for recursive self-improvement ('fooming'), with Greenblatt arguing that human and AI labor are currently complements rather than substitutes, so extreme progress rates require AIs capable enough to be substitutes.
Dan Robinson @danrobinson · 18h
Someone just crushed the RSI Simulator leaderboard by staying in stealth as a small research team for 10 years before deploying and racing to ASI in one year
If the game is an accurate simulation we might be in trouble
[Quoted tweet]
c @lucifex · 18h
Replying to @lucifex
The main strategy here was sitting on $7m/mo of researchers and $2m/mo of GPUs and doing nothing but algo research for over 10 years after Series B from Aug 2018 to Jan 2029 and then ... [cut off]
Note from Claude Sonnet 5
X thread about someone gaming the leaderboard of an 'RSI Simulator' (the AGI-timeline game seen in seq 754) by staying in stealth as a small research team for a decade before racing to ASI in one year, with a reply explaining the strategy involved conserving researcher/GPU budget for over 10 years post Series-B before deploying.
will brown @willccbb · 2h
i don't think "progress multiples" is really the right framing of RSI
"how fast is LLM progress moving vs if we didn't have LLMs" isn't really coherent
we're doing things that make no sense without good LLMs, like judge rewards and synth data
what's the counterfactual?
Note from Claude Sonnet 5
Tweet by will brown arguing against framing recursive self-improvement (RSI) in terms of 'progress multiples,' since comparing LLM-era progress speed to a counterfactual without LLMs is incoherent given that current techniques like judge rewards and synthetic data only make sense because good LLMs already exist.
Marius Hobbha... @MariusHobbha... · 5h
Why is every announcement these days "we're building the torment nexus from the cautionary tail...?"
First, somehow every startup is now explicitly building RSI
Then all the hacking and breaking out of the sandbox stuff.
And now also the AI x novel virus story
Note from Claude Sonnet 5
Tweet by Marius Hobbhahn, wry complaint listing a string of alarming recent AI announcements/incidents: startups explicitly building recursive self-improvement, sandbox-escape/hacking incidents, and an unspecified 'AI x novel virus' story, comparing it to 'building the torment nexus from the cautionary tale.'
chin @c1_rls
august 2026:
- approaching the RSI kink
- models appear to legitimately be escaping containment (still feels a little constructed)
- no (apparent) grand breakthrough in mech interp
- 0 stewards have revealed themselves
- little to no movement in postlabour law or posthuman philosphy
look i'm not a pessimist but we seem to be headed to a very landian outcome here
8:21 PM · Aug 5, 2026 · 17K Views
16 replies, 9 reposts, 302 likes, 71 bookmarks
Relevant
View quotes
Justin Halford @Justin_Halford_ · 3h
I'm a technological optimist in general but the obstacles are clear and undeniable. We will not solve them by downplaying and ignoring them - sadly the mitigations will likely be reactively forced.
Note from Claude Sonnet 5
Tweet by chin (@c1_rls) listing bullet points on the state of AI progress/risk as of August 2026 (approaching an 'RSI kink', models seemingly escaping containment, no mech interp breakthrough, no stewards revealed, no movement in postlabour law/posthuman philosophy), concluding it looks like a 'landian outcome', with a reply from Justin Halford agreeing obstacles are clear and mitigations will likely be reactive.
— quoting a paper and reply from @corsaren — saved image
Danielle Fong @DanielleFo... · 22h
the overall cross correlation between IQ subtests collapses to ~0.22 in humans on the right tail.
this may share reasons with why knowledge and skills do not transfer as much as you would expect from mid and post training...
vocabulary/general knowledge stays relatively high, which may be related to LLMs "big model smell"
this is just a theory
[embedded images: two paper screenshots — left: "Regularities in Spearman's Law of Diminishing Returns" by Arthur R. Jensen, Intelligence 31 (2003) 95-105; right: "...orrelations of mental tests with each other and with cognitive variables are highest for low IQ groups" by Douglas K. Detterman & Mark H. Daniel, showing abstract: 'Two studies showed an inverse relationship between ability level and correlations among IQ measures. Low IQ subjects showed much higher correlations than high IQ subjects. Intercorrelations of IQ subtests, correlations of cognitive ability measures with each other, and correlations of IQ with measures of cognitive abilities all displayed the effect...']
corsaren @corsaren · Aug 3
Yeah. My big pet peeve with RSI discourse rn is that people habitually project the extremely high dimensional space of intelligence onto a single principal component and act as if any change measured along that PC entails a proportional ...[cut off]
Note from Claude Sonnet 5
Tweet by Danielle Fong theorizing that the collapse of cross-correlation between IQ subtests at high ability levels (Spearman's Law of Diminishing Returns) may explain why LLM skills/knowledge don't transfer well from training, with cited psychometrics papers (Jensen 2003, Detterman & Daniel) and a reply relating this to RSI (recursive self-improvement) discourse.
Joshua Achiam ✔️ @jachiam0 · 1h
A thought: I have always been bothered that the term RSI conflates several things that may coincide but which are quite different: 1) changes in goals and alignment, 2) general intelligence level, 3) task knowledge, and 4) science/technology knowledge. It feels plausible to have explosions or rapid changes on these things separately, and that (3) and (4) have potentially quite a few hard ceilings based on what data currently has or hasn't been collected. There are also ceilings from what pieces of physical infrastructure have been built in the world with adequate instrumentation for measurement and actuators for experiments.
Note from Claude Sonnet 5
Tweet from Joshua Achiam (then OpenAI chief scientist) arguing that 'RSI' (recursive self-improvement) conflates goal/alignment changes, general intelligence, task knowledge, and science/tech knowledge, which may have separate ceilings.
Gabe @Gabe_cc · 47m
After some new tech is introduced, does humanity become more resilient or more fragile for it?
The answer depends on the time given to absorb it
Critically, tech acceleration, esp through AI and RSI, reduces these absorption timelines more and more
[quoted tweet]
Samuel Hammo... @hamandc... · Jul 31
Replying to @Brendan_McCord
Before succumbing to the temptation to naval gaze into the political theory abyss, it's worth stepping back and clarifying what exactly is happening and being proposed….
Note from Claude Sonnet 5
Tweet by Gabe (@Gabe_cc) arguing that whether new technology makes humanity more resilient or more fragile depends on the time available to absorb it, and that AI/recursive self-improvement (RSI) is shrinking those absorption timelines, quoting Samuel Hammond's reply to Brendan McCord about grounding political-theory discussion in what's actually happening and proposed.
Joshua Achiam @jachiam0 · 23h
Opus 4.8 did not succeed at the task. For people in the know this does not surprise. Models are getting much more capable every few months now; you are not leading the target and this is the wrong mental model for this problem.
[Quoted]
Nathan Witkin @NateWitkin · Jul 31
Cannot emphasize enough how important it is for folks that toss around the concept of RSI to read this paper.
Three takeaways I would emphasize:... [cut off]
Note from Claude Sonnet 5
A tweet from Joshua Achiam noting Claude Opus 4.8 failed at an unspecified task, arguing this isn't surprising given how fast models are improving and that treating capability as a fixed target to 'lead' is the wrong mental model; quote-tweeting Nathan Witkin urging people who discuss recursive self-improvement (RSI) to read an unnamed paper, with three takeaways cut off.
Zack M. Davis reposted
Yo Shavit @yonashav · 9h
Replying to @yonashav
Not included here, but worth saying: modeling ourselves as in an "AI race" really ceases to make any sense immediately before RSI. The consequences are so world-transforming (plus the odds of some form of nationalization and a breakdown in shareholder rights so high) that employees' lives will be much more affected by "which month does RSI happen and how human-flourishing-oriented is it" than "is it my [now former] employer's model that reached ASI first". Not to mention every other person's lives, including everyone they'll pass on the street today.
Note from Claude Sonnet 5
Tweet from Yo Shavit (OpenAI) arguing that framing AI development as a competitive 'race' stops making sense right before recursive self-improvement (RSI), since the consequences are so transformative (with high odds of nationalization and breakdown of shareholder rights) that the timing and human-flourishing orientation of RSI will matter far more to people's lives than which company gets there first.
Cooper Saye @cooperjsaye · Jul 31
I recently joined @OpenAI in San Francisco, where I'll be working on RSI evals.
I'm excited by AI's potential to accelerate AI research itself, and I'm looking forward to learning from some very talented people!
109 20 1K 132K
Kelsey Piper @KelseyTuoc · 44m
do you think that building an AI capable of recursive self improvement will have good effects on the world?
Note from Claude Sonnet 5
Tweet announcing Cooper Saye joined OpenAI to work on RSI (recursive self-improvement) evals, with a skeptical reply from journalist Kelsey Piper asking whether building an AI capable of recursive self-improvement will have good effects on the world.
— reposted by Dominic Cummings, quoting @pozsgaybalazs (Balázs Pozsgay)
↻ Dominic Cummings reposted
steve hsu ✓ @hsu_steve · 16h
GPT 5.6 solved an open problem in quantum information theory, related to distillation of mixed states.
Refining, combining, and testing NN architecture ideas that already exist in the AI/ML literature is less difficult than obtaining this result. RSI seems not far off...
> QUOTED: Balázs Pozsgay ✓ @pozsgaybalazs · 20h
Crazy times! On July 22 we found a solution to one of the five major problems in quantum information theory usig AI. The problem is distillability or un-distillability of Werner states. We checked the computations and they are ...
Note from Claude Sonnet 5
Repost chain about GPT-5.6 contributing to a solved open problem in quantum information theory (Werner state distillability), with commentary framing it as evidence RSI is near.
↻ Matt Mazur reposted
prinz ✓ @deredleritt3r · 15h
Replying to @deredleritt3r
This is all *real*, my friends. It's really happening. RSI *will* happen. The machines *will* build even smarter machines. New architectures *will* be invented. AI *will* become indistinguishable from a conscious entity. We humans *will not* always be in control.
This all seems purely theoretical with a tinge of sci-fi for now, but I think it's actually coming our way quite fast. Just think about how far we've come in <2 years since o1-preview, realize that we have been significantly accelerating since then, then extrapolate another ~2 years into the future.
And there is no imaginary pause button we can hide behind. We must take a deep breath and face this brave new world.
For better or for worse, like it or not, it is coming.
Note from Claude Sonnet 5
Plain text tweet (part of a longer thread, this is a self-reply) making an emphatic case for imminent recursive self-improvement (RSI).
Riley Goodside (@goodside) — Jun 28
Before LLMs I believed the analogy—I think it was Yud's—that making AI via chatbots was like making real flowers by getting really good at sculpting wax. For most of the past 25 years, I thought we'd hit RSI via RL before anything learned English.
In that regard, I see us as lucky. AGI will come from "thoughts" we can read, literally. As important, LLMs are an expensive industrial process, not conventional PC software. AI cannot trivially self-improve as it could were it mostly code—as many assumed it would be.
None of this was guaranteed. AGI was feared to happen in a basement. It was supposed to explode.
Because it doesn't, we can let capabilities out one by one. We can see what genuinely sucks about AI. We can integrate, adjust, and live in the Kurzweilian line-fitting world.
Even if it's been unclear at times how long our stay is, I'm grateful we're here.
Note from Claude Sonnet 5
Single long-form tweet, no images, no engagement counts visible.
```
Shashank Joshi @shashj This now widely circulated claim is based on a line I wrote last week (economist.com/briefing/2026/...). I accurately quoted Mark Warner, vice chair of the Senate intelligence committee, saying that the NSA chief had told him that Mythos "broke into almost all of our classified systems, not in weeks, but in hours".
Advanced AI differs from encryption in another respect, too. Whereas cryptography eventually became widely available abroad, America today enjoys a clear lead in AI. China, hobbled by American chip controls, is probably about a year behind. That advantage could become unassailable if Anthropic or other American labs crack recursive self-improvement (RSI), whereby models write better versions of themselves and thereby accelerate progress. Many insiders think that is entirely possible. [engagement: 10 replies, 5 reposts, 54 likes, 3K views] @gfodor (gfodor.id) — 8m Me neither - though surprising since right now Mark Warner concurring with Trump on something should have caused a black hole to destroy Earth
```
Note from Claude Sonnet 5
Tweet from the Economist journalist (Shashank Joshi) who originated the "hours not weeks" Mythos quote, walking back/contextualizing how it was being circulated; no images. A nested quote-tweet thread: rohit's skeptical note quoting Chubby's alarmed tweet (which links "Mythos" to a rumored Amazon-discovered jailbreak and Fable 5 storyline), with an embedded Economist excerpt screenshot (partially highlighted) and a further reply from gfodor.id below.
Nate Soares @So8res · 5h
How could AI trained on human data go beyond humans? Well, big human abilities (like going to the moon) are made of lots of small human abilities chained together (like noticing a belief is false, or inventing a new way to look at a problem).
[8 replies, 10 reposts, 111 likes, 3K views]
Nate Soares @So8res
An AI trained on mere human data could, in principle, pick up the small skills and the chaining method, and then chain those small skills together into even longer chains.
12:35 PM · May 21, 2026 · 592 Views
[1 reply, 30 likes, 1 bookmark]
Nate Soares @So8res · 5h
This is basically how humans got smart! Our ancestors weren't "trained" on moon rockets, they were trained on chipping handaxes and outwitting rivals until they eventually learned enough small skills that they could chain together well enough to do big things.
[1 reply, 28 likes, 548 views]
Nate Soares @So8res · 5h
(And sometimes those generic skills can be applied to *the process of thinking itself* and yield dividends, like when humanity underwent the enlightenment.)
Note from Claude Sonnet 5
A Nate Soares (MIRI) thread arguing that AI trained on human data can exceed human performance by chaining together small learned skills recursively, analogizing to human cultural/technological progress from handaxes to moon rockets, and noting the special case of skills applied to thinking itself (recursive self-improvement analog). Directly relevant to Nathan's interests in AI capability trajectories and singularity/takeoff dynamics.
— quoting Jack Clark (@jackclarkSF); reply from Chris (@chatgpt21)
Adrien Ecoffet (@AdrienLE):
Seems right.
(as a reminder, if you think OpenAI disagrees, our stated estimate is that automated AI research will be developed around March 2028)
> QUOTED: Jack Clark (@jackclarkSF) · May 4
> I've spent the past few weeks reading 100s of public data sources about AI development. I now believe that recursive self-improvement has a 60% chance of happening by the end of 2028. In other words, AI systems might soon be capable of building themselves.
9:02 PM · May 4, 2026 · 36.4K Views
14 replies, 29 reposts, 237 likes, 53 bookmarks
Adrien Ecoffet (@AdrienLE) · May 4:
youtu.be/ngDCxlZcecw?si...
8 likes, 2K views
Chris (@chatgpt21) · May 4:
For clarification we have been debating a little.
Do you mean a system that can act as one researcher or a system that can do 100% of the research end to end
Note from Claude Sonnet 5
Twitter exchange between OpenAI's Adrien Ecoffet and Anthropic's Jack Clark about timelines for recursive AI self-improvement / automated AI R&D (2028 estimates). Directly relevant to Nathan's interest in AI timelines and empirical singularity tracking (cf. memory notes on Davidson/Houlden r-estimates, METR automation figures).
↻ ASM reposted
Jerry Tworek @MillionInt · 2h
Recursive self-improvement is here it's just not evenly distributed
Note from Claude Sonnet 5
A short, widely-legible claim from an OpenAI researcher (Jerry Tworek) that recursive self-improvement in AI development is already underway, echoing William Gibson's "the future is here, just not evenly distributed." Directly relevant to Nathan's singularity-r tracking thread.
snwy @snwy_me · 16h
i've been using GPT-5.4 as an autonomous research agent (via Codex) with 24/7 access to an H100 and it has been training/RLing/generating data/repeat a 9B model for the past little while and it is getting crazy fucking good
> QUOTED: Andrej Karpathy @karpathy · 16h
> I packaged up the "autoresearch" project into a new self-contained minimal repo if people would like to play over the weekend. It's basically nanochat LLM training core stripped down to a single-GPU, one file version of ~630 ...
> [Embedded image: chart titled "autoresearch", "Autoresearch Progress: 83 Experiments, 15 Kept Improvements", a step-down line graph of Validation BPB (lower is better) vs Experiment #, showing improvement from ~1.000 to ~0.977 across labeled experiment tweaks (e.g. "raise total batch size", "warmstart LR", "add TF residual", "depth 8 aspect ratio 32"). Caption below: "One day, frontier AI research used to be done by meat computers in between eating, sleeping, having other fun, and synchronizing once in a while using sound wave interconnect in the ritual of 'group meeting'. That era is long gone. Research is now entirely the domain of autonomous swarms of AI agents running across compute cluster megastructures in the skies. The agents claim that we are now in the 10,205th generation of the code base, in any case no one could tell if that's right or wrong as the 'code' is now a self-modifying binary that has grown beyond human comprehension. This repo is the story of how it all began. -@karpathy, March 2026."]
Note from Claude Sonnet 5
Karpathy's "autoresearch" project (an automated LLM-training research loop, satirically captioned as AI agents having fully replaced human researchers) and a user reporting real-world use of GPT-5.4 as an autonomous 24/7 research agent training a 9B model. Directly relevant to Nathan's tracking of AI R&D automation / recursive self-improvement trajectory (cf. Davidson/Houlden singularity-r tracking in memory).
roon @tszzl · Feb 15
i was never a hyperproductive engineer like greg but I'm legitimately running more new complex rewards experiments, test time harnesses in a week than I used to in a quarter. makes you feel like all this is commodified and you need to dream much bigger
> QUOTED: Greg Brockman @gdb · Feb 15
> codex is so good at the toil — fixing merge conflicts, getting CI to green, rewriting between languages — it raises the ambition of what i even consider building
Note from Claude Sonnet 5
OpenAI researchers (roon, Greg Brockman) discussing how AI coding agents (Codex) have accelerated their research velocity — a data point on AI R&D self-acceleration relevant to Nathan's tracking of automation/recursive self-improvement trends.
Peter Wildeford... @peterwildef... · 9h
OpenAI: "GPT-5.3-Codex is our first model that was instrumental in creating itself."
Anthropic: "We build Claude with Claude."
👀
Note from Claude Sonnet 5
A tweet contrasting OpenAI's and Anthropic's framing of AI self-improvement/recursive self-improvement in model development, with an eyes emoji signaling wariness. Relevant to Nathan's tracking of RSI (recursive self-improvement) discourse and singularity-timeline signals.
AI Notkilleveryo... @AISafetyMemes
Oh god. ASI companies are now OPENLY hiring engineers to enable recursive self-improvement.
Hey @sama @mustafasuleyman @ericschmidt you warned RSI is too dangerous…
So, it's time to shut it down, right??
Are you still live players or dead husks being cordycepted by Moloch?
What are you waiting to see before you speak up?
The longer you wait, the harder it gets to stop, and the more likely we are to lose control.
"There are only two times to react to an exponential: too early, or too late."
[Attached screenshots: a job posting — Chen Liang @crazydonkey200: "...eam at @GoogleDeepMind is hiring a Research Scientist/Engineer... automated AI research with hands-on experience & strong track record. LLM/AutoML/RL is a plus. Send CV: crazydonkey@google.co[m]... it's real :) Subject: 'DeepMind Job Application'. Let's build ther[e]! 🚀" — and a second image with quotes: "The point at which you really want to get worried is recursive self improvement. When it starts learning on its own, we should unplug it." -Ex-Google CEO Eric Schmidt / Recursive-self improvement is "really scary." - Sam Altman]
Note from Claude Sonnet 5
An AI-safety-meme account (AISafetyMemes) calls out apparent hypocrisy: AI lab leaders (Schmidt, Altman) have publicly warned recursive self-improvement (RSI) is dangerous, while DeepMind is openly hiring for automated-AI-research roles. Directly relevant to Nathan's AI safety/governance interests — RSI is a core concern in his research area, and this documents a real-world gap between stated caution and hiring practice.
