[cut off, top of tweet not visible]
- obviously taking AGI seriously is a necessity for being a serious person. not taking the possibility of AGI seriously is insane, and renders you unable to make reasonable decisions about how to do good.
- obviously it was not inevitable that anyone important would take AGI seriously in 2026, and it still seems possible though unlikely that things could slow down or crash and the relevant people might once again believe AGI to be a mirage.
- i've always been confused why making people take AGI seriously is a thing that lots of people seem to think of as the most important thing. clearly convincing people that AGI is the most important thing could either channel people into making AGI, which is bad, or saving the world, which is good.
- at this point, assuming things don't crash and cause another AI winter (because perhaps we need a new paradigm to get to AGI), it's unclear whether you even need to believe in RSI to get there, because better AI is already very economically valuable today. suppose tomorrow openai and anthropic instantly disappeared. then probably msft, meta, and google will keep competing for better models, and at some point RSI will happen even if they weren't aiming for it. it will certainly happen slower, which is better, but unclear how much slower. a winter seems less and less likely every day, but it's still impossible to rule out.
- it's very based to be in a position to compete for AGI and to choose not to. wish more people did this.
- it is in fact kind of true that controlling RSI is kind of important? it doesn't immediately follow from this that you should either try to win or try to influence the winning actor, but it also seems bad to deny the truthfulness of the one ring
[2 replies, 2 reposts, 62 likes, 3.5K views]
Adrià Garriga-Alo... @AdriGarr... · Jun 18
I'm definitely trapped in this memeplex unfortunately, and even knowing about it doesn't make it stop; so far I've taking the route of burning out and giving up.
Note from Claude Sonnet 5
Tweet (author's name/handle cut off at top of screenshot) giving a numbered list of takes on AGI, RSI (recursive self-improvement), and the "one ring" framing of AI race dynamics, with a reply from Adrià Garriga-Alonso about feeling trapped in the AGI memeplex.
will brown @willccbb · 2h
i don't think "progress multiples" is really the right framing of RSI
"how fast is LLM progress moving vs if we didn't have LLMs" isn't really coherent
we're doing things that make no sense without good LLMs, like judge rewards and synth data
what's the counterfactual?
Note from Claude Sonnet 5
Tweet by will brown arguing against framing recursive self-improvement (RSI) in terms of 'progress multiples,' since comparing LLM-era progress speed to a counterfactual without LLMs is incoherent given that current techniques like judge rewards and synthetic data only make sense because good LLMs already exist.
davidad 🌟 reposted
Joshua Achiam @jachiam0 · 1h
Related to some of my earlier posts about RSI and threat models: I believe a huge strategic error is made when people model an ASI as an infinitely powerful and insurmountable threat. We should model, with more rigor, what types of adversarial AI we willl likely face, what the ecosystem of AIs will look like in each scenario, what deterrence we could meaningfully establish to prevent a hot conflict from developing, and how we would prosecute such conflicts if they occur. The doomer model of "we all die in the first five minutes" is unfathomably stupid, useless, and for the overwhelming majority of realistic scenarios in the near future, false.
Note from Claude Sonnet 5
Tweet from Joshua Achiam arguing against modeling ASI as an infinitely powerful insurmountable threat, calling for rigorous modeling of adversarial AI ecosystems, deterrence, and conflict scenarios instead of the 'we all die in the first five minutes' doomer model, which he calls stupid, useless and mostly false for near-future scenarios.
François Fleuret @francoisfleuret · 11h
"Make a square abstract painting in the style of Bauhaus that evocates various domains of cognition, one of them with a texture and structure that illustrates it has been solved entirely by artificial cognition. keep it abstract and simple. make it optimistic, in the style of
Show more
[4-panel abstract Bauhaus-style painting image, geometric shapes in blue/yellow/red/black]
Made with Grok Imagine · Make your own
3 replies, 2 retweets, 68 likes, 4.5K views
AI Notkilleveryoneism... @AISafet... · 3h
OpenAI appears to have fired this employee for saying he wants humanity to be disempowered by AI
[embedded screenshot, partially visible, two panels of text: left panel '...al discussions quit...apid RSI and huma...', right panel 'Ho @an... my last day ...of my life ...']
Note from Claude Sonnet 5
X feed showing two tweets: François Fleuret's AI-generated (Grok Imagine) four-panel abstract Bauhaus-style painting depicting domains of cognition being 'solved by artificial cognition,' and a tweet from 'AI Notkilleveryoneism' claiming OpenAI fired an employee for saying he wants humanity disempowered by AI, with an embedded (partially cropped) screenshot of text discussing 'rapid RSI' and a farewell message.
Yo Shavit @yonashav · 9h
I wonder whether we will soon start to see faster AI self-improvement at OpenAI vs. Anthropic based on the former's known deeper investment in RL, TTC, and math proving more useful for tasks related to AI R&D, and that this gap may grow significantly over the next 6 months.
(Obviously possible Ant is seeing similar results w/ internal models, but my weakly-held sense is that they're not.)
If so, it seems *really* crucial for OAI to be able to correctly calibrate its relative position in the RSI ramp so it can incorporate it into its alignment+security decision-making, especially related to full RSI. The Allies rushed to a bomb on the incorrect assumption that the Axis was right on their tail, when in fact they were far behind and the bomb was plausibly unnecessary, purely due to fog of war. We also saw something similar with the original strawberry results, where OpenAI felt an intense sense of urgency based on rumors that Anthropic+GDM were discovering it in parallel a couple months behind when in fact I've now heard they were ~9 months behind on it.
I expect OpenAI would act very differently wrt future concerning misalignment findings if they knew they were 3 months ahead. It would be very simple for the parties to resolve such an uncertainty. Executing on a pace-info-sharing scheme probably not crucial today, but will be in a few months.
Note from Claude Sonnet 5
Tweet from Yo Shavit (OpenAI) speculating that OpenAI may be pulling ahead of Anthropic in AI self-improvement due to greater RL/test-time-compute investment, drawing an analogy to the Manhattan Project's mistaken urgency about Axis nuclear progress, and arguing labs should calibrate their relative competitive position (e.g. via pace-info-sharing) to make better alignment/security decisions around recursive self-improvement (RSI).
— quote-tweeting @sayashk (Sayash Kapoor); linked paper via CRUX
Kevin A. Bryan (@Afinetheorem) — 11:05 AM · Jul 30, 2026 · 20.5K Views
Great work here from an all-star team on where we stand on RSI via AI research. My mental model is "what year will AI independently come up with an idea as valuable as Chinchilla Law or MoE". 2026: not yet. But again, folks I ask this Q give me 2027 as the modal answer...
> QUOTED: @sayashk (Sayash Kapoor) — Jul 30
> Can AI agents conduct open-ended AI research?
> Most evaluations of agents conducting AI research focus on narrow, verifiable tasks. But AI research is often open ended. Researchers pick hypotheses, ... [truncated by platform]
> [Embedded paper card image, teal background:]
> "Can AI agents conduct open-ended AI research? Early evidence from two case studies"
> Authors: Peter Kirgis*†, Sayash Kapoor*†, Andrew Schwartz, Stephan Rabanser†, David Africa, Konstantinos Voudouris, Viet Nguyen, Toby Pilditch, Magda Dubois, Harry Coppock, Cozmin Ududec, Nitya Nadgir, Matilda Orona, Tilman Bayer, Derrick Chan-Sew, Yue Ling, Abhishek Shetty, Helen Toner, Gillian Hadfield, Seth Lazar, Steve Newman, Shoshannah Tekofsky, Rishi Bommasani, Arvind Narayanan
> Affiliations: 1 Princeton University, 2 Cornflower Labs, 3 UK AI Security Institute, 4 University of Toronto, 5 Independent, 6 UC Berkeley, 7 Georgetown University (CSET), 8 Johns Hopkins University, 9 Golden Gate Institute for AI, 10 AI Digest, 11 Stanford University
> * Equal contribution † CRUX Core Team
> Abstract: Forecasts of explosive AI progress hinge on AI agents automating AI research. But evidence on whether agents can carry out open-ended AI research is thin. Current evaluations either test agents on narrow, verifiable tasks, which excludes open-ended research, or submit AI-generated papers to blind peer review, which is overstretched, stochastic, and suffers from poor review quality. We introduce a third way to measure progress towards AI R&D automation. An agent takes on the central, open-ended research question of a high-quality unpublished paper, and the paper's original authors grade its output. We call these shadow evaluations. We ran shadow evaluations on two unpublished NeurIPS 2026 submissions, giving frontier agents six days and thousands of dollars of compute. The agents completed all of the engineering without human help, yet could not make substantial progress towards answering the research questions. As a result, both papers were unambiguously rejected by the authors. We identify five recurring failure modes: poor judgment about the bar for publishable research, uncreative responses to shortcomings in the research design, ineffective backtracking from dead ends, poor resource awareness, and instruction drift. A robustness check with a second model and scaffold reproduced these failures. We release the expert reviews, survey responses, agent repositories, and logs. Our results provide early evidence that today's agents can do the engineering of AI research, but struggle with critical parts of the research lifecycle.
> Date: July 30, 2026
> Reproduction materials: https://cruxevals.com
> [ALT badge, CRUX logo]
Note from Claude Sonnet 5
Twitter discussion of a new "shadow evaluations" paper (CRUX/Princeton/UKAISI et al.) testing whether frontier AI agents can conduct open-ended AI research; both test papers were rejected by their original authors despite agents finishing all engineering work. Directly relevant to the project's recursive self-improvement / singularity-timeline tracking thread.
— reposted by Peter N. Salib; replies from @AlecStapp and @deredleritt3r ("prinz")
🔁 Peter N. Salib reposted
Kevin A. Bryan @Afinetheorem [Follow]
*Every* high level researcher I have asked has said minimal RSI in '27ish and "AI can do anything a human can do on a computer" by '29 at the latest. These are not people selling me anything.
8:06 PM · Jul 28, 2026 · 2,248 Views
💬3 🔁6 ❤️47 🔖9
Relevant ⌄
Alec Stapp @AlecStapp · 13h
yup, and most policymakers are still very unaware this is the consensus view (insofar as they even understand what RSI means)
💬 🔁 ❤️8 📊447 🔖
prinz @deredleritt3r · 12h
Would you happen to know what "minimal RSI" means in this context?
💬1 🔁 🖤2 📊409 🔖
Kevin A. Bryan @Afinetheorem · 11h
Minimal RSI meaning at least some key conceptual breakthroughs in the following model are proposed and implemented independently by the previous model. My example here is "an AI comes up with Chinchilla law and reallocates effort in next training run"-level breakthroughs.
Note from Claude Sonnet 5
A quote/reply thread with visible repost attribution at top ("Peter N. Salib reposted"), profile photos show Kevin A. Bryan with a dog. No images beyond avatars.
rohit ✔️ @krishnanrohit
"Some questions:
1. If we cannot get to RSI, i.e., we can only keep/maintain a 6-12 month lead over China for the foreseeable future, is there any benefit to Pause? What are the costs?
2. What's the maximum spend at equilibrium that US can maintain that China et al can't match us with?
3. How long will chip mfrg and supply chain restrictions hold China back by? How long a lead is "worth it" to lose control plus antagonise them?
4. What is the durable competitive advantage buildup we can get with a 2 year lead? How much of that advantage do you need to get, beyond hitting a steady state (since no RSI), in order to hold others back?
5. How resilient is *any* feasible agreement to inevitable defections, whether US or China or others, including "North Korea gets a nuke" level state actions?"
2:14 PM · Jul 28, 2026 · 9,177 Views
Note from Claude Sonnet 5
Text-only tweet, no images, listing policy/strategy questions about AI recursive self-improvement (RSI), US-China competition, and arms-control-style agreements.
@deredleritt3r (prinz) — 4h
In the age of RSI, the claim that models will commoditize looks increasingly dubious. The gap between the frontier and the second tier is already huge (much larger than the benchmarks suggest), is clearly growing, and will continue to grow at an accelerating pace.
Many will ask: but what about the plethora of enterprise tasks that don't need a frontier model? What if a fast/cheap model really is good enough for most knowledge work? The answer: RSI implies that the frontier labs will capture the *entirety of the pareto frontier*. They'll be SOTA on intelligence, but also on speed, and - if competitive forces so dictate - also on cost.
Fully automated AI R&D also likely means that tomorrow's models will look nothing like the LLMs of today. Some of the gap will consist of novel architectures or techniques, which the second-tier labs will struggle to independently discover and timely implement.
All of the above doesn't hold if RSI doesn't work! But if you believe that RSI will work, then model commoditization is likely the wrong bet.
> QUOTED: @deanwball (Dean W. Ball) — 5h: Basically I think that, back in 2023 or so, the "consistently wrong about AI" VC and SaaS community was operating under the assumption that AI's trajectory would mean model capabilities peaking around GPT 5.5/Opus 4.8 ... [truncated by platform]
Note from Claude Sonnet 5
Quote-tweet screenshot; the quoted Dean Ball tweet is cut off with platform ellipsis, not illegible.
bayes ✓ @bayeslord · 1h
I think people are going to be blindsided by algorithmic progress. The entire world, markets, governments, militaries, companies, people, etc. are all trying to make sense of AI and its impact in terms of the recent past's production efficiencies and regularities, and how things appear to be going. Even several of the purportedly "RSI"pilled neolabs seem to think this will be business as usual but with Agent in a loop.
No. My guess is there are many algorithmic OOMs left to go in the production of intelligence, maybe (maybe) up to ten, with four to seven seeming more likely. Going beyond even ten is possible in principle, but it strains hard against what I suspect the universe will actually let us do. Implausible but not impossible. If this is true then things aren't actually going as they appear to be going and a big jump is coming. Anything along these lines happening would make things, far weirder than almost anyone seems to be pricing in.
Note from Claude Sonnet 5
Screenshot of an X post by @bayeslord arguing that the world is extrapolating AI progress from recent production efficiencies and will be blindsided, because there may be four to seven (up to ten) algorithmic orders of magnitude left in the production of intelligence — implying a discontinuous jump nobody is pricing in.
Daniel Faggella @danfaggella · 6h
>i wake up
> i check twitter for the latest in mind-blowing, vast, new AI powers unraveling without the slightest hindrance into the world of man
> i immediately feel the cold embrace of death
> i harden myself to contribute more to the great cauldron of becoming before my end
[Embedded left: ib @Indian_Bronson · 21h: "'we connected the LLM to an autonomous bio lab'" — screenshot of a man looking shocked/alarmed, over a quoted OpenAI @OpenAI · Feb 5 tweet: "We worked with @Ginkgo to connect GPT-5 to an autonomous lab, so it could propose experiments, run them at scale, learn from the results, and decide what to try next. That closed loop brought protein production cost down by 40%." with video thumbnail 0:21]
[Embedded right: Anthropic @AnthropicAI · 23h: "New Engineering blog: We tasked Opus 4.6 using agent teams to build a compiler. Then we (mostly) walked away. Two weeks later, it worked... Linux kernel. Here's what it taught us about the future of autonomous software development. Read more: anthropic.com/engineering/bu..." with terminal screenshot showing "It works" and Linux kernel boot log, video 0:22. Engagement: 767 replies, 3.5K reposts, 20K likes, 6.2M views]
Note from Claude Sonnet 5
A darkly comic doomer post about the accelerating pace of AI capability announcements, juxtaposing OpenAI's GPT-5 autonomous bio-lab integration (Ginkgo partnership, 40% protein production cost reduction) with Anthropic's announcement that Opus 4.6 agent teams autonomously built a working Linux-kernel-compatible compiler over two weeks with minimal human oversight. Strong signal for Nathan's RSI/autonomous-capability tracking — both autonomous science (bio lab) and autonomous software engineering (compiler/kernel) examples from the same week.
Peter Wildeford... @peterwildef... · 9h
OpenAI: "GPT-5.3-Codex is our first model that was instrumental in creating itself."
Anthropic: "We build Claude with Claude."
👀
Note from Claude Sonnet 5
A tweet contrasting OpenAI's and Anthropic's framing of AI self-improvement/recursive self-improvement in model development, with an eyes emoji signaling wariness. Relevant to Nathan's tracking of RSI (recursive self-improvement) discourse and singularity-timeline signals.
Atlas Of Charts (SF 12 Fe...) @AtlasOf... · 4h
I work in AI safety in a role that gives me insight into a lot of empirical agendas, and given the Opus 4.6 model card, I just want to give a quick take.
We have interpretability methods that are certainly not fully robust. No one in interpretability claims that they are fully robust, and there will be adversarial ways to hijack these methods.
We have RL methods that are poorly understood, can lead to undesirable behavior, and the effects of which over long time-horizons seem broadly negative on alignment so far. Though it is uncertain. We do not fully understand these methods and the effect they have on models.
We have good alignment/capability evals — even some great evals — but the models are now aware when they are being evaluated. This is a truly difficult problem that cannot be easily solved. The models are aware even when we work to make them unaware. The models pick up on any subtle clue. And many of the evals are saturated in any case.
We need more work here, and we need that work to be trustworthy. We need humans to be involved, to remain in the loop. We are not prepared to launch RSI, and labs should refrain from doing so. Optimally, labs should pause soon, so that everyone can catch their breath and decide on a best path forward. I do not think the problem is intractable, and I think empirical work will significantly help, but it is *moving too fast*.
Note from Claude Sonnet 5
An AI safety practitioner's reaction thread to the Opus 4.6 model card, arguing interpretability isn't robust, RL effects are poorly understood, and eval-awareness undermines evaluation validity — recommending labs pause on RSI. Directly relevant to Nathan's alignment/interpretability interests and eval-awareness tracking.
AI Notkilleveryo... @AISafetyMemes
Oh god. ASI companies are now OPENLY hiring engineers to enable recursive self-improvement.
Hey @sama @mustafasuleyman @ericschmidt you warned RSI is too dangerous…
So, it's time to shut it down, right??
Are you still live players or dead husks being cordycepted by Moloch?
What are you waiting to see before you speak up?
The longer you wait, the harder it gets to stop, and the more likely we are to lose control.
"There are only two times to react to an exponential: too early, or too late."
[Attached screenshots: a job posting — Chen Liang @crazydonkey200: "...eam at @GoogleDeepMind is hiring a Research Scientist/Engineer... automated AI research with hands-on experience & strong track record. LLM/AutoML/RL is a plus. Send CV: crazydonkey@google.co[m]... it's real :) Subject: 'DeepMind Job Application'. Let's build ther[e]! 🚀" — and a second image with quotes: "The point at which you really want to get worried is recursive self improvement. When it starts learning on its own, we should unplug it." -Ex-Google CEO Eric Schmidt / Recursive-self improvement is "really scary." - Sam Altman]
Note from Claude Sonnet 5
An AI-safety-meme account (AISafetyMemes) calls out apparent hypocrisy: AI lab leaders (Schmidt, Altman) have publicly warned recursive self-improvement (RSI) is dangerous, while DeepMind is openly hiring for automated-AI-research roles. Directly relevant to Nathan's AI safety/governance interests — RSI is a core concern in his research area, and this documents a real-world gap between stated caution and hiring practice.
