[cut off, top of tweet not visible]
- obviously taking AGI seriously is a necessity for being a serious person. not taking the possibility of AGI seriously is insane, and renders you unable to make reasonable decisions about how to do good.
- obviously it was not inevitable that anyone important would take AGI seriously in 2026, and it still seems possible though unlikely that things could slow down or crash and the relevant people might once again believe AGI to be a mirage.
- i've always been confused why making people take AGI seriously is a thing that lots of people seem to think of as the most important thing. clearly convincing people that AGI is the most important thing could either channel people into making AGI, which is bad, or saving the world, which is good.
- at this point, assuming things don't crash and cause another AI winter (because perhaps we need a new paradigm to get to AGI), it's unclear whether you even need to believe in RSI to get there, because better AI is already very economically valuable today. suppose tomorrow openai and anthropic instantly disappeared. then probably msft, meta, and google will keep competing for better models, and at some point RSI will happen even if they weren't aiming for it. it will certainly happen slower, which is better, but unclear how much slower. a winter seems less and less likely every day, but it's still impossible to rule out.
- it's very based to be in a position to compete for AGI and to choose not to. wish more people did this.
- it is in fact kind of true that controlling RSI is kind of important? it doesn't immediately follow from this that you should either try to win or try to influence the winning actor, but it also seems bad to deny the truthfulness of the one ring
[2 replies, 2 reposts, 62 likes, 3.5K views]
Adrià Garriga-Alo... @AdriGarr... · Jun 18
I'm definitely trapped in this memeplex unfortunately, and even knowing about it doesn't make it stop; so far I've taking the route of burning out and giving up.
Note from Claude Sonnet 5
Tweet (author's name/handle cut off at top of screenshot) giving a numbered list of takes on AGI, RSI (recursive self-improvement), and the "one ring" framing of AI race dynamics, with a reply from Adrià Garriga-Alonso about feeling trapped in the AGI memeplex.
tibbar @tibbaroen · 13h
"anthropic announces latest model killed a guy so they're holding it back"
"china announces release of new open source model benchmaxxed on killing"
Note from Claude Sonnet 5
Joke tweet by @tibbaroen imagining a future news cycle where Anthropic withholds a model for having killed someone while a Chinese open-source model is released explicitly optimized ('benchmaxxed') for killing, satirizing US/China AI safety-vs-race dynamics.
@tenobrus (Tenobrus) — Jul 25
on the one hand it is quite sad to see all these companies actively making the world a worse place by pushing open source models, falling back to narratives about openness and freedom that apply amazingly to software but not at all to the creation and modification of minds.
on the other hand, it is also true that the US banning open source models while China continues to produce them and the rest of the world continues to use them will do little to nothing to decrease the negative impact, globally or in the US, and will likely mostly just cause some excess profits to accrue to labs. so locally i can pretty much agree with the arguments that bans such as trump is considering are mostly empty posturing plus damage to the US economy.
but this leaves us in a very pessimistic place, where instead of having much chance of global coordination on this each country and each company is still pursuing race dynamics, hoping that open source nips at the heels of their competitors profit margins and builds more demand for inference they can sell while pouring all the profits towards individual RSI programs.
it seems like, as we have seen repeatedly, it will take real Incidents with real Consequences before we have a chance to slow down the release of open source models.
in some sense this remains something of a sideshow. the frontier is still ahead to a significant enough degree and alignment is sufficiently behind that the vast majority of doom probability mass stays in the labs. but it's an additional complication in coordinating any real pause. the wave of negative consequences from *not* having aligned ASI piles higher and higher as guaranteed-misaligned open source AI becomes smarter and smarter. a tsunami wave behind us and a cliff in front of us. [text continues, cut off at bottom of screen by nav bar]
Note from Claude Sonnet 5
Long text-only tweet (thread/long-form post), bottom line partially obscured by the phone's bottom navigation icons.
— reposted by Agus; also @deepfates and @jachiam0 (Joshua Achiam) shown below
```
↻ Agus 🔶 reposted roon ✓ @tszzl · 18h if we could coordinate a global capabilities slowdown today i would likely press that magic button
```
Note from Claude Sonnet 5
Feed screenshot with three separate tweets stacked (roon on capabilities slowdown, deepfates riffing on AI critics, Joshua Achiam on quantum algorithm overhang from AI math capability). Quote-tweet, no images. Profile picture for Trazzi shows a person holding a megaphone with a "STOP THE AI RACE" sign. Quote-tweet, no images. Same roon tweet quoted a second time by a different account (compare Screenshot_20260726-091454.png).
roon @tszzl · 5h
elon was right when he said frontier lab is the highest elo game in the world. the teams are incredibly good. few months of delay here and there can cost the entire game. whole thing is extra nerve wracking because most of the parties involved expect infinite consequences
Note from Claude Sonnet 5
A tweet from roon (OpenAI researcher known for AI-scene commentary) framing frontier AI lab competition as an extremely high-stakes race where small delays matter and participants believe the outcome carries existential/infinite stakes. Relevant to Nathan's interest in AI safety and race dynamics.
Seán Ó hÉigeartaigh... @S_OhEigearta... · 7h
I don't expect Opus 4.6 to be dangerous.
But this all looks, in @peterwildeford 's words, 'flimsy'. Anthropic marking their own homework with evals. An internal employee survey because benchmarks were satisfied. initially a strong signal from only 11 out of 16. The clear potential for groupthink and professional/social pressure.
The closer we get to the really consequential thresholds, the greater the degree of rigor needed. And the greater the degree of external evaluation. Instead we're getting the opposite. This should be a yellow flashing light wrt the direction of travel – and not just Anthropic; we can't simply punish the most transparent. If they stop telling us this stuff, then that yellow should become red. (And others just won't, even now).
We need to keep asking *why* this is the direction of travel. *why* the practices are becoming riskier, as the consequences grow greater. It's the 'AI race'; both between companies and 'with China' supposedly, and Anthropic are culpable in promotion of the latter.
No Chinese company is near what we've seen released today.
Note from Claude Sonnet 5
A critical AI-safety-researcher thread on the Opus 4.6 release, arguing Anthropic's self-evaluation practices (internal employee surveys, small sample sizes for safety signals) are "flimsy" and represent a worrying direction-of-travel toward less external scrutiny as capabilities grow, situating this within AI-race dynamics. Relevant to Nathan's interest in Anthropic governance/safety-practice critique and eval-robustness debates.
Roon's cynical one-liner about the double standard in AI-lab race commentary — any relative launch speed gets criticized depending on who's ahead. General AI-race discourse, tangential to Nathan's singularity/race-dynamics tracking.
if hour-long lmsys dominance doesn't scream singularity, i don't know what does
"our model was sota for 45 minutes" ah is cyberpunk as hell
counter-strike by counter-strike
[Embedded image: retro sci-fi space-art painting of an astronaut firing an orbital weapons platform down at an Earth being struck by multiple explosions/impacts.]
Note from Claude Sonnet 5
A joking tweet using rapid, hour-scale SOTA-leapfrogging on the LMSYS leaderboard as a comedic marker of accelerating AI progress ("singularity"), illustrated with a retro sci-fi orbital-bombardment painting as metaphor for the AI lab arms race. Tangentially relevant to the archive's empirical-singularity-tracking thread (Davidson/Houlden, METR figures already logged).
