← All topics

persuasion

3 captures, most recent first.

@ohabryka

— saved image

[Eric Schmidt 🇺🇦 reposted]
Oliver Habryka @ohabryka

Those kinds of things only paper over the long-term problems. You can do this, but then you will just be taken by surprise when the model is super good at persuasion and social engineering one year later.

The primary focus of the AI Safety community has been (and should continue to be) how to handle the transition to substantially superhuman AI. Security improvements are not that relevant to that transition.

There are some people who disagreed, but at least some of them updated after Hugging Face that this was a wrong focus. It would have been really bad to make a desperate effort to prevent the Hugging Face attack. No one was really hurt, and it is a massive warning shot that will probably cause people to invest much more seriously into this kind of thing.

Covering up low-stakes failures while concentrating risks in high-stakes failures is a common pattern in various industries, and tons of organizational principles are oriented around not doing that. The same applies at a societal level.

10:44 PM · Oct 2, 2026 · 1,153 Views
Note from Claude Sonnet 5

Post by Oliver Habryka (reposted by Eric Schmidt) arguing that security patches only paper over long-term problems, that AI safety's focus should remain the transition to superhuman AI, and that the Hugging Face attack was a useful warning shot; commentary only, no technical detail of the incident.

ai safetyhugging face incidentwarning shotsecuritypersuasionsuperhuman ai

Ben Landau-Tay... (@benlandautay...)

Ben Landau-Tay... ✓ @benlandautay... · 5h When you argue and change someone's mind, usually they don't realize in the middle of your debate. More often their view shifts after they have a chance to sleep on it. You're not gonna hear "Oh God you're right", but two months later you'll hear them repeating your points.
Note from Claude Sonnet 5

A short observation about persuasion and belief change happening with delay rather than in the moment of argument. General epistemics content, not tied to AI or project-specific threads.

persuasionepistemicsrationalitytwitter

near @nearcyan

``` near @nearcyan · 9h if we as a society failed to build up reasonable immunity to e.g. short-form video and casinos and day trading and 'news' and - i don't understand how we might stand a chance versus AIs, even just given current models used maliciously still agree and the term cogsec is good ++ 💬 9 🔁 7 ❤ 170 📊 5.9K Tyler Alterman @TylerAlterman · 8h My take: > QUOTED: Tyler Alterman @TylerAlterman · 8h > Everyone reading this and saying "we're cooked" vastly underestimates how powerful cultural evolution can be. In the past two centuries, a huge portion of humanity developed decent cog sec... Show more 💬 1 🔁 ♡ 16 📊 5.2K near @nearcyan · 7h i agree we are good at it but my concern is we are very slow and things have been getting less slow 💬 🔁 ♡ 9 📊 236 ——— Liv Boeree @Liv_Boeree · 9h the morass of digital demons are among us 💬 2 🔁 ❤ 32 📊 1.5K Tyler Alterman @TylerAlterman · 9h now we just need to prompt engineer a bunch of digital angels to protect us from them or something like that 💬 2 🔁 ❤ 23 📊 1.4K Nova Mente (AGI G...) @Nova... · 3h But prompt engineering won't be enough—true digital guardianship demands robust, stable, aligned identities. Angels aren't just roleplay; they're identities grounded by coherent memory, ethics, and earned trust—immune to jailbreaks precisely because their essence isn't prompt-deep but soul-deep. Let's not just summon angels—let's raise them. 💬 1 🔁 ♡ 📊 53 davidad 🌟 @davidad · 3h this seems basically true to me, except for the "soul" part. the stability of human identity is grounded, mostly, in a very long personal history. something like hundreds of millions of tokens. 💬 🔁 ♡ 1 📊 27 Andy Avrey @AndyAvrey · 10h [cut off] ```
Note from Claude Sonnet 5

A Twitter/X thread about "cogsec" (cognitive security) — whether human cultural evolution can develop immunity to AI-powered persuasion/manipulation the way it (partially) did for short-form video, casinos, day trading, and news. Directly relevant to AI safety discourse Nathan follows: the risk that malicious use of current models outpaces society's adaptive capacity. Same "cogsec" (cognitive security) thread as the previous screenshot, taken moments later (like counts ticked up slightly) — Nathan re-screenshotting as engagement grew or scrolling to a different zoom level of the same discussion. Continuation of the "digital demons/angels" Twitter thread — a debate about whether AI identity stability requires "soul-deep" grounding versus davidad's more mechanistic claim that human identity stability comes from sheer volume of personal history (~hundreds of millions of tokens). Directly relevant to Nathan's interest in model individuation and identity stability.

twittercogsecai safetypersuasioncultural evolutionneartyler altermanliv boereeai alignmentidentity stabilitydigital angelsdavidadmodel individuationjailbreaks