dave kasten @David_Kasten . 12h
One genuinely sad, complicated, awful thing I experienced at defcon -- the very concept of a cool bug, of a bold and creative CTF hack, is going away.
We're in the foothills of the singularity now, and even the most impressive hackers on planet Earth are starting to be outstripped by AI.
I feel...well, I feel very confused and awful about this. I've loved how @defcon feels like a celebration of human vigor and freedom, a space for creative, clever, playful people to do their best work.
And now, John Henry is being outworked by the steel-driving machine. We're just in a place where LLMs can Do The Job, as well as humans can.
I suspect that for a brief moment, this will be a shining period of uplift for some hackers, as their ambitions and capabilities rise higher than ever before. But for so many others, a door is swinging closed.
And that's something melancholy. No denying it.
[quoted tweet]
s1r1us in sf 🏝️ @S1r1u5_ . Aug 12
i talked to a lot of people who played the defcon ctf finals to understand how llms affected them, both professionally and mentally.
i went in hoping to learn that human intuition ... [cut off]
Note from Claude Sonnet 5
Tweet from Dave Kasten reflecting on DEF CON, mourning that AI is starting to outstrip the best human hackers at CTF-style bug-finding, quote-tweeting s1r1us in sf's thread about interviewing DEF CON CTF finalists on how LLMs affected them.
ai capabilitiesdefconhackingsingularityautomation of expertise
dave kasten @David_Kasten · 2h
If you saw this inside a SCIF, you'd very very quietly lock your screen and walk as fast as you could without drawing attention to the security office.
Note from Claude Sonnet 5
Tweet by Dave Kasten, apparently quote-tweeting or replying to an unseen image/post, joking that seeing 'this' inside a SCIF (a secure government facility) would prompt someone to quietly lock their screen and alert security.
securitytwitterscif
dave kasten ✅ @David_Kasten — 14m
One thing that is genuinely surprising to me is that many folks who are indeed very smart DC national security folks really have not priced this in, even though they've fully updated on Mythos having significant cyber capabilities.
(Source: conversations over the past several weeks)
> QUOTED: prinz ✅ @deredleritt3r — 2h
> If you think the USG's involvement with Anthropic and OpenAI models is bad now, just wait until the next generation of models trips up thresholds for bio and chemical risk.
Note from Claude Sonnet 5
Quote-tweet exchange about US national-security awareness of AI cyber/bio/chem risk, referencing the "Mythos" model.
ai national securitycyber capabilitiesbio riskus governmentclaude mythos
dave kasten ✓ @David_Kasten
The intelligence law section of this is very persuasive if you don't realize that every bad intelligence scandal in the last 30 years had a legal memo saying it complied with those authorities
> OpenAI ✓ @OpenAI · 18h
> Yesterday we reached an agreement with the Department of War for deploying advanced AI systems in classified environments, which we requested they make available to all AI companies.
> ...
> Show more
Readers added context
OpenAI's contract forbids use of their technology in autonomous weapons where "law, regulation, or Department policy requires human control." However, current DoW policy does not require human approval for an autonomous weapon to use force.
openai.com/index/our-agre...
warontherocks.com/2025/05/autono...
Note from Claude Sonnet 5
OpenAI's official announcement (quoted) confirming the Department of War classified-environment deployment deal referenced in adjacent screenshots, plus a skeptical reply from an intelligence-community-adjacent commentator, and a Community Notes correction noting the "human control" safeguard is toothless because current DoW policy doesn't actually require human approval before autonomous weapons use force. Substantively important documentation of the actual OpenAI-DoW agreement's loophole, directly relevant to AI governance/autonomous weapons tracking.
twitteropenaidepartment of warautonomous weaponsai policyhuman controlclassified aidave kasten
```
dave kasten (@David_Kasten, 2h): "I would also like to register a piece of _advice_: If you work on AI issues, treat this as a fire drill. Think about, for good or ill, the ways in which you have or have not tracked and responded to this well today, and figure out what to do better next time." > QUOTED: Dean W. Ball (@deanwball, 3h): "registering the prediction that moltbook will probably not become Actually Important, even if it does become a viral phenomenon covered in mainstream media. it'll be a neat curiosity, maybe even a continued source of intrigue,
entertainment, and controversy, but not itself be some earth-shattering thing. yet it is a big deal. it is not even so much what it reveals about what will be possible in the future that matters so much (that has been obvious for years to most careful observers of this field). instead, what matters is what this phenomenon reveals *to whom* about what is going to be possible in AI, and what is possible now. it's "normies," a horrible word by which I mean "people not obsessed with AI," waking up to, well, the reason all of us are obsessed with AI. in that sense it is a little like DeepSeek, though perhaps on a much smaller scale (hard to know, but probably), which brought in many new people to the field and caused others to start taking AI much more seriously. on the whole this is good for AI and for society, though it may provoke some startled and therefore rash reactions. > QUOTED: Nabeel S. Qureshi @nabeelqu · 8h > Moltbook (the new AI agent social network) is insane and hilarious, but it is also, in Nick Bostrom's phrase, a Disneyland with no children > [Embedded image of text, apparently from Bostrom, reading:] We could imagine, as an extreme case, a technologically highly advanced society, containing many complex structures, some of them far more intricate and intelligent than anything that exists on the planet today – a society which nevertheless lacks any type of being that is conscious or whose welfare has moral significance. In a sense, this would be an uninhabited society. It would be a society of economic miracles and technological awesomeness, with nobody there to benefit. A Disneyland with no children.
```
Note from Claude Sonnet 5
AI-policy commentators (Dave Kasten, Dean Ball — both known for AI governance work) discussing MoltBook (the AI-only social network where the "Crustafarianism" religion emerged, per the companion screenshot from the same session) as a "fire drill" for the AI policy community — a small, low-stakes but structurally interesting event worth treating as practice for tracking and responding to emergent AI phenomena. Directly connects to Nathan's MoltBook-adjacent interests (the corpus referenced in his uniqueness_checker work) and to AI governance discourse on how seriously to take viral AI-agent behavior. Commentary thread on Moltbook (the AI-agent social network Nathan's own project archive references, e.g. `moltbook_instructions.md`) reacting to Nabeel Qureshi's framing of it via Bostrom's "Disneyland with no children" thought experiment about consciousness and moral significance. Directly relevant to model welfare/consciousness threads in Nathan's archive.
moltbookai-governanceai-policyemergent-behaviortwitterdean-balldave-kastenai agentsconsciousnessmoral significancebostromdean ballnabeel qureshiai commentary
