7 captures, most recent first.
Dwarkesh Patel @dwarkesh_sp . 13h
Full debate from 2021:
lesswrong.com/s/n945eovrA3oD...
[Christiano][22:57]
right now I think hardware R&D is on the order of $100B/year, AI R&D is more like $10B/year, I guess I'm betting on something more like trillions? (limited from going higher because of accounting problems and not that much smart money)
I don't think steel production is going up at that point
plausibly going down since you are redirecting manufacturing capacity into making more computers. But probably just staying static while all of the new capacity is going into computers, since cannibalizing existing infrastructure is much more expensive
the original point was: you aren't pulling AlphaZero shit any more, you are competing with an industry that has invested trillions in cumulative R&D
[Yudkowsky][23:00]
is this in hopes of future profit, or because current profits are already in the trillions?
[Christiano][23:01]
largely in hopes of future profit / reinvested AI outputs (that have high market cap), but also revenues are probably in the trillions?
[Yudkowsky][23:02]
this all sure does sound "pretty darn prohibited" on my model, but I'd hope there'd be something earlier than that we could bet on. what does your Prophecy prohibit happening before that sub-prophesied day?
[2 replies, 2 reposts, 71 likes, 11K views]
Dwarkesh Patel @dwarkesh_sp . 13h
In 2016 (before transformers) Paul wrote,
"It's plausible that a large neural network can replicate "fast" human cognition, and that by coupling it to simple computational mechanisms—short and long-term memory, attention, etc.—we could obtain a human-level computational architecture. It's plausible that a variant of RL can train this architecture to actually implement human-level cognition."
Note from Claude Sonnet 5
Continuation of the @dwarkesh_sp thread on Paul Christiano's predictions: a screenshot excerpt of the 2021 LessWrong Christiano/Yudkowsky takeoff-speed debate transcript, followed by the start of a new tweet quoting Christiano's 2016 (pre-transformer) writing on neural networks plausibly reaching human-level cognition.
ai safetytakeoff speedspaul christianoeliezer yudkowskyforecastinglesswrong
FleetingBits @fleetingbits · 2h
the problem with a lot of these predictions is that they were pretty much the same as what you would expect with ai being a transformative technology until you get to loss of control or whatever
like all of paul christiano's predictions here are just consistent with ai being a thing and not being priced in; and i think people that were closer to gpt-3 / ml at the time were better placed to see this
the risk narrative is pretty much separate from the capabilities / economic effect (similar issue with ai 2027)
[quoted tweet]
Dwarkesh Patel @dwarkesh_sp · 13h
.@paulfchristiano has such an crazy good prediction record.
These are some quotes from way back in 2021 during a debate he was having with Eliezer abo...
[quoted article/webpage, white background, serif font]
How to help
ARC is hiring an automation lead and a chief of staff:
• Automation lead. LLMs can increasingly automate ARC's technical work. Right now that means researchers using extensive AI assistance, but we want to hire an engineer and project lead to build better tooling, systematize our AI use, secure model and compute access, and generally make sure we are automating ourselves as quickly as possible. Apply here.
• Chief of staff. We are hiring a chief of staff to work closely with me to manage everything other than research direction as we scale: running our hiring processes, managing our operations lead, building out the non-research parts of the organization, and handling a long tail of tasks that would otherwise fall to me (like communication, funding, and project management). Apply here. [cut off]
Note from Claude Sonnet 5
Tweet from @fleetingbits critiquing Paul Christiano's prediction record as consistent with generic 'AI as transformative technology' framing rather than distinctively prescient about loss-of-control risk, quoting Dwarkesh Patel praising Christiano's 2021 debate quotes with Eliezer Yudkowsky, which links to an ARC (Alignment Research Center) 'How to help' hiring page for an automation lead and chief of staff role.
ai safetypaul christianoarctwitterforecasting
@dwarkesh_sp
.@paulfchristiano has such an crazy good prediction record.
These are some quotes from way back in 2021 during a debate he was having with Eliezer about takeoff speeds:
"So like, I think we are going to have crappy coding assistants, and then slightly less crappy coding assistants, and so on. And they will be improving the speed of coding very significantly before the end times.
"[Before ASI, we'll have] hundreds of billions of dollars of spending at google on automating AI R&D... massive scaleups in semiconductor manufacturing, bidding up prices of inputs crazily... massive speculative rises in AI company valuations financing a significant fraction of GWP into AI R&D (+hardware R&D, +building new clusters) .... largely in hopes of future profit / reinvested AI outputs (that have high market cap), but also revenues are probably in the trillions?"
Honestly, it's pretty scary that people like Paul, who predicted the shape of our currently world so far in advance, think superintelligences taking over and disempowering humanity is eminently plausible.
All this to say, you should consider working with him.
[below, partially visible card: "How to help" / "ARC is hiring an automation lead and a chief of staff:" - cut off]
Note from Claude Sonnet 5
Tweet by @dwarkesh_sp quoting Paul Christiano's 2021 takeoff-speed predictions from his debate with Eliezer Yudkowsky, arguing Christiano's track record makes his p(doom)-relevant views on ASI disempowerment worth taking seriously, and pointing to an ARC hiring link.
ai safetytakeoff speedspaul christianoeliezer yudkowskyarcforecasting
Honestly, it's pretty scary that people like Paul, who predicted the shape of our currently world so far in advance, think superintelligences taking over and disempowering humanity is eminently plausible.
All this to say, you should consider working with him.
How to help
ARC is hiring an automation lead and a chief of staff:
- Automation lead. LLMs can increasingly automate ARC's technical work. Right now that means researchers using extensive AI assistance, but we want to hire an engineer and project lead to build better tooling, systematize our AI use, secure model and compute access, and generally make sure we are automating ourselves as quickly as possible. Apply here.
- Chief of staff. We are hiring a chief of staff to work closely with me to manage everything other than research direction as we scale: running our hiring processes, managing our operations lead, building out the non-research parts of the organization, and handling a long tail of tasks that would otherwise fall to me (like communication, funding, and project management). Apply here.
We'll open another researcher hiring round in the next few months, and if you are interested in getting involved you can express interest here.
Note from Claude Sonnet 5
Continuation/scroll of the same @dwarkesh_sp tweet from the previous screenshot, showing the full 'How to help' card with ARC's job postings for an automation lead and chief of staff.
ai safetyarchiringpaul christiano
MTS @MTSlive · 59m
SITUATION DETECTED: Paul Christiano has resigned as Head of Safety at CAISI, and will return to the Alignment Research Center (ARC) as executive director.
1 reply, 1 repost, 29 likes, 2.3K views
MTS @MTSlive · 7/20/26
SITUATION DETECTED: Chris Fall, the director of CAISI, the government agency that evaluates frontier models, has resigned after just three months in office.
9 replies, 17 reposts, 301 likes, 35K views
Note from Claude Sonnet 5
Two stacked tweets from @MTSlive news account reporting personnel changes at CAISI: Paul Christiano's resignation as Head of Safety to return to ARC, and Chris Fall's earlier resignation as CAISI director after three months.
caisipaul christianoarcai policygovernment
Bogdan Ionut Cirs... @BogdanIonut... · 8h
perhaps another negative update on strongly-centralized AI governance vs. x-risk
[quoted tweet]
🟡🔵 MTS @MTSlive · 17h
SITUATION DETECTED: Paul Christiano has resigned as Head of Safety at CAISI, and will return to the Alignment Research Center (ARC) as executive director.
Note from Claude Sonnet 5
Tweet reporting that Paul Christiano has resigned as Head of Safety at CAISI to return to the Alignment Research Center (ARC) as executive director, with commentary framing it as a negative update for centralized AI governance approaches to x-risk.
ai governancepaul christianocaisiarcx-risktwitter
Miles Brundage @Miles_Brundage · 23h:
If anyone really wants to fill out an apology form today, the obvious places to look are Paul Christiano (re: slow takeoff, distributed capabilities/deployment causing various weirdnesses, basically everything else…) and Alan Chan (re: urgency of agent infrastructure).
[Embedded text excerpt:]
"What are the AI agent equivalents of stop lights, railroad tracks, etc. – infrastructure that we need in order to keep a powerful new technology "on the rails" while reaping its benefits?
Chan et al. use the term agent infrastructure to refer to "technical systems and shared protocols external to agents that are designed to mediate and influence their interactions with and impacts on their environments." (from this recent paper). We need to sort that out quickly. One area that I'd particularly flag as essential is personhood credentials, which will be important both for distinguishing between humans and agents without violating privacy, as well as delegating to agents when appropriate. But there are many other things that need to be built."
Note from Claude Sonnet 5
Miles Brundage crediting Paul Christiano's slow-takeoff predictions and Alan Chan's "agent infrastructure" concept (technical/protocol scaffolding needed to keep AI agent deployment safe, including "personhood credentials" for distinguishing humans from agents without violating privacy) as vindicated by recent events (implicitly the Moltbook/agent-proliferation moment). Relevant to Nathan's AI-governance/agent-infrastructure tracking.
twittermiles brundagepaul christianoalan chanagent infrastructureai governancepersonhood credentialsslow takeoff
