@XRobservatory

An often misunderstood concept is the offense defense balance of existential risk. It is used below by @GavinSBaker who writes that "we can simply keep AIs in check with a balance of power between many AI systems, as we do with humans. I believe this is the best path forward". Such a positive offense defense balance would be amazing. As Yann Lecun would put it: my good AI will defeat your bad AI. Indeed, this would be everyone's favorite solution, if only it would work. As always, before talking about solutions, we should detail which risk scenario we're talking about exactly, because the offense defense balance is likely scenario-dependent. We will go over a few important ones. For biorisk, offense looks like a bad or careless actor creating a pandemic, while defense would take shapes such as vaccines and early warning systems, presumably powered by AI. Of course, the latter two are amazing to have, but who would say: let covid #2 rip, because we have shiny new vaccines? Preventing pandemic creation, rather than trusting that things will turn out fine if only everyone has advanced AI to make themselves a good vaccine, seems common sense here. (Since @DarioAmodei seems to worry a lot about pandemics, this may be his thinking). For the classical Bostrom-style AI takeover scenario, offense defense balance is unfortunately generally believed to be pro-offense as well. One could easily see that many not particularly aligned superintelligent agents might want to kindly share the world amongst themselves, but would all agree to remove all humans first. Ants do not particularly benefit from our democratic system, as we tend to not give them a vote. Some other reasons why the offense defense balance may be negative are here: https://lesswrong.com/posts/LFNXiQuGrar3duBzJ/what-does-it-take-to-defend-the-world-against-out-of-control… Recently, other risk scenarios have become more popular: Paul Christiano's what failure looks like (likely positive offense defense balance), gradual disempowerment (no offense defense balance), and takeover by collusion (such as in IABIED, maybe positive offense defense balance). In general: important decisions such as whether AGI should be free and open weight or controlled by governments, depend on the specifics of which risk scenarios we want to guard against and what the offense defense balance for these scenarios is. Rather than stating vibes-based takes, we should do research into these topics and translate the results into policy where needed. > **Gavin Baker @GavinSBaker** · 2026-08-15 > > Sholto, thank you for setting the record straight. Larger issue is that multiple very serious people in Silicon Valley have heard some variation of this and believe it to be true. And the reason it is believable to so many is that it is consistent with Dario’s public messaging --- @ylecun seems to stay on message over the years > **Yann LeCun @ylecun** · 2026-08-16 > > For about 10 years now, I have argued that the \*only\* way forward is for AI technology to be widely available, shared, and open. > > Like the printing press and the Internet, AI amplifies human intelligence and efficiency by improving access to knowledge. > > To empower individuals, --- ##### Comments > **Lars Holm Tjessem @t4intelligence** · [2026-08-18](https://x.com/t4intelligence/status/2089635119027921094) > > This is the key point: there is no single offense–defense balance for AI. > > In bio, the asymmetry may be especially dangerous. Defensive AI must detect, attribute and contain threats repeatedly. An offensive actor may only need to succeed once. > > “Good AI will stop bad AI” is therefore not a safety strategy by itself. The relevant question is: under which threat models does defense actually have the structural advantage?