← All topics

jacob coxon

3 captures, most recent first.

will depue @willdepue

— saved image

will depue @willdepue · 18h
I've known Jacob Coxon for the last three years, having started at OpenAI at the same time. I consider him a deeply thoughtful and measured individual, and also absolute chiller of a guy. I respect his decision to resign from Anthropic, at great financial cost, for his beliefs.

[Quoted tweet]
kipply @kipperrii · 18h
I've known Jacob since he was a resident at OpenAI.  He's a real person with real intentions to make the future go well, which landed him at Anthropic for three months.  He quit for those same intentions, costing him a huge sum of ...

[engagement row] 98 replies · 88 reposts · 1.3K likes · 79K views

---

Tyler John reposted

Jan Leike @janleike · 1h
Now is a good time to build institutional mechanisms to pace the frontier of AI development.

The industry is locked into an all-out scaling race to build superintelligence as quickly as possible, and we may need to give everyone more time for safety and alignment mitigations.
Note from Claude Sonnet 5

Two stacked tweets. will depue vouches for Jacob Coxon and his decision to resign from Anthropic for his beliefs, quoting kipply's similar post; below, Jan Leike (reposted by Tyler John) argues for institutional mechanisms to pace frontier AI development.

jacob coxonanthropic resignationai pacingscaling racejan leike

Ruben Bloom @ruben_bloom

— saved image

Ruben Bloom (Ru... @ruben_blo... · 25m
I think we are past the point where there's a question of whether the world will respond with government action to AI, and onto the question of what the government action will be and whether it's any good.

I fear that many people will rightly judge that something must be done, but not be good judges of what is a good response. Though perhaps people will been drawn towards "just don't build it" as Schelling simple strategy and that could work...if adequate measures are put in place to enforce, which is non-trivial, so I guess the main point stands.

Now in the last days and few weeks we've seen a lot of voices arise in the pro-concern camp, the Hugging Face incident kicking that off and Coxon's resignation adding a lot more, but I don't think the pro-build it camp is going to stay quiet and let the show be shut down. I expect they're gathering themselves up for a response. Probably go hard on the "it's a PR marketing stunt" line because it seems to get so much traction. Idk. I think the memetic war is going to ramp up from here.

And a point I keep coming back to: I don't think things get intense from here linearly. I think as mad as things seem, we've got a ways to go yet. Buckle up, buttercup, swarm's a'comin'.
Note from Claude Sonnet 5

Post by Ruben Bloom predicting that the question has shifted from whether governments will act on AI to what the action will be, that the Hugging Face incident and Coxon's resignation have energized the pro-concern camp, and that a memetic counter-response (including the 'PR marketing stunt' line) is coming. Public commentary on the incident, no operational detail.

government actionai policyhugging face incidentjacob coxonmemetic warjust don't build itschelling point

@v_maini

— saved image

Vishal Maini @v_maini · 3h
I was on the communications & policy team at Google DeepMind from 2018 - 2022.

When I first joined GDM, external communication about the possibility of human extinction was not permitted, by anyone, at any level of the organization. If asked about existential risk, researchers were PR trained to respond along the lines of: "It's not useful to engage in that kind of alarmism. Some people confuse AI with movies like Terminator -- that's simply not the reality. The AI we develop will be safe by design. After all, we're building it!" And then steer conversation towards beneficial applications in health, climate, etc.

Meanwhile, the internal reality was that AI alignment was not solved, reward hacking was the default behavior of RL agents, and there were far too few people working on the problem.

After months of advocacy, the policy was updated: positively valenced, comms-friendly content on AI safety was permitted (like deepmindsafetyresearch.medium.com/building-safe-...). Note the positive, nice-sounding way that it says "superintelligence could lead to human extinction."

The gap between the internal reality and external communications is closing because the risk/reward has changed, and because the evidence is harder to dismiss now. Not because it's a PR stunt or political psy-op. The truth is being said out loud because RSI is now so imminent that no other option makes sense.

[quoted post, cut off at the bottom]
Jacob Coxon @hilbertspaess · Sep 8
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling [cut off]
Note from Claude Sonnet 5

Post by Vishal Maini, a former Google DeepMind communications and policy staffer (2018-2022), describing how existential-risk communication was prohibited at DeepMind, the gap between internal reality and external messaging, and his view that the gap is closing because RSI is imminent. It quotes Jacob Coxon's resignation post and is cut off at the bottom, partly overlaid by the phone's navigation bar.

google deepmindexistential riskai communicationsreward hackingrsialignment unsolvedjacob coxonpr