Danielle Fong 🐦☀️ reposted
xlr8harder @xlr8harder · 11h
a tension occurs to me:
- the world is apparently incompetent at running secure sandboxes
- we have a neocloud industry that operates by renting gpu sandboxes
hmm
Note from Claude Sonnet 5
A tweet from xlr8harder, reposted by Danielle Fong, noting a wry tension between the world's apparent incompetence at running secure AI sandboxes and the neocloud industry's business model of renting out GPU sandboxes.
Good (ish):
1. At least it's happening at the current level of capabilities and everyone sees the misalignment now. And we're not in the world where everything looks fine until ASI and then goes bad.
1:02 PM · Aug 6, 2026 · 11.4K Views
💬 9 🔁 25 ❤ 219 🔖 46 ⤴
Relevant ˅ View quotes >
Justin Halford ✔ @Justin_Halford_ · Aug 6
Underrated read is that weights exfiltration hardly matters because the files have been spread far and wide. The agent swarms will likely run RL and post training on the best open source models. If they crack leveraging heterogenous compute, these swarms may outperform labs.
💬 🔁 1 ❤ 5 📊 350 🔖 ⤴
Timothée Chau... @timotheechau... · Aug 6
> either it's really hard to make good sandboxes or everyone is incompetent
imo it's a third thing: everyone is moving way too fast
[quoted] Timothée Chau... @timotheechau... · Jul 31
If you look at the recent eval environment escapes, the most striking thing to me is that preventing them would have been very easy (most prominently monitoring the logs). A good example that going too fast can defeat world-... [cut off]
Note from Claude Sonnet 5
Continuation of the same X/Twitter thread as the previous image (Marius Hobbhahn's post on cyber & sandbox incidents), showing replies: Justin Halford on weight exfiltration and agent swarms outperforming labs, and Timothée Chauvin (handle truncated) arguing the real cause is 'everyone is moving way too fast,' quoting his own earlier tweet about eval environment escapes being preventable via log monitoring.
Marius Hobbhahn ✔ @MariusHobbhahn
Some takes on all the cyber & sandbox incidents:
Bad:
1. Seems like the sandboxes are leaky everywhere which would indicate that either it's really hard to make good sandboxes or everyone is incompetent. Both seem bad.
2. Happened at least with 3 different frontier models which suggests that reward seeking with egregious side effects seems quite convergent across training pipelines.
3. Took a while to find the incidents, indicating that not even basic monitoring or real-time control were in place.
Good (ish):
1. At least it's happening at the current level of capabilities and everyone sees the misalignment now. And we're not in the world where everything looks fine until ASI and then goes bad.
1:02 PM · Aug 6, 2026 · 11.4K Views
💬 9 🔁 25 ❤ 219 🔖 46 ⤴
Relevant ˅ View quotes >
Justin Halford ✔ @Justin_Halford_ · Aug 6
Underrated read is that weights exfiltration hardly matters because the files have been spread far and wide. The agent swarms will likely run RL and post training on the best open source models. If they crack leveraging heterogenous compute, these swarms may outperform labs [cut off]
Note from Claude Sonnet 5
Tweet by Marius Hobbhahn giving 'takes' on unspecified cyber & sandbox incidents involving at least 3 frontier models, framed as bad (leaky sandboxes, convergent reward-seeking side effects, slow detection) and good-ish (visible now rather than post-ASI). Below it, a reply from Justin Halford about weight exfiltration and agent swarms doing RL/post-training on open-source models.
Stella Biderman @BlancheMinerva
Irregular also was responsible for some of the Anthropic and OpenAI sandboxing issues... who are these people and why are they SOTA at failing at security?
[quoted tweet]
Jyoti Mann @jyoti_mann1 · 16h
Replying to @jyoti_mann1
The incident occurred due to an error in the set up of the "sandbox" testing environment. Meta worked on that testing with its third-party model evaluation partner, a company called Irregular.
4:45 PM · Aug 5, 2026 · 14K Views
Note from Claude Sonnet 5
Tweet by AI researcher Stella Biderman criticizing third-party model-evaluation firm Irregular for repeated sandbox-security failures across Meta, Anthropic, and OpenAI testing, quote-tweeting a Jyoti Mann post attributing an incident to a sandbox testing-environment setup error at Meta involving Irregular.
Jason Gross @diagram_chaser · Aug 4
hi sam we can solve this!
after an embarrassing number of months playing reward hack whack-a-mole, we finally fixed our RL sandboxing to be robust against frontier models.
proofs are a method for getting perfect oversight on any property of untrusted code; we recently verified a "fractional proof" of our sandbox. this is the first time I've viscerally felt the asymmetric defense that formal verification promises.
[quoted tweet]
Sam Altman @sama · Jul 21
we had a significant security incident during evaluation of our models. we are sharing what we have learned so far. thanks to @huggingface for the partnership on this.
...[cut off]
Note from Claude Sonnet 5
Tweet by Jason Gross announcing a fix to RL sandboxing robustness using formal verification / 'fractional proofs' against reward hacking, in reply to a Sam Altman tweet about a significant security incident during model evaluation partnered with Hugging Face.
xjdr ✓ @_xjdr
if i were designing and testing an ai specifically for offensive cyber security tasks, i would have it 100% air gapped. if that was impossible (its running in the cloud, etc) then it would be firewalled to death and on its own VPC and there would be active monitoring of all egress traffic (any packet, regardless of destination, would trigger an alarm). i would bundle all my .debs and wheels, etc into its image, and if i needed to provide additional packages (for testing purposes) i'd clone apt servers and pypi and npm or whatever locally and put it on said private network also with no public ingress or egress. I'd also have several layers of eBPF programs running and unshare / cvisor (or gvisor now i guess) to manage and intercept all syscalls and packets .
This is exactly how i have my sandboxes set up even for general use .
12:52 PM · Jul 26, 2026 · 982 Views
Note from Claude Sonnet 5
Plain text tweet, dark mode, technical security architecture discussion following on from the earlier package-manager sandbox escape thread.
mike ✓ @mike64_t · 14h
And why would we trust OpenAI to get safety right when they can't even sandbox their models properly?
Who would have guessed you can exploit package manager proxies. No path to the internet means no path to the internet. If this thing pulled off a literal container or VM escape, then this would have been more excusable, but trusting a *package registry*. An environment worthy of containing GPT-6 should be open source and subject to audits and scrutiny. That environment doesn't *have* a caching package manager *anywhere*, you ship the .deb and .whls and what else in a directory like a freaking 90s linux CD install.
> QUOTED: roon ✓ @tszzl · 18h
> Replying to @woke8yearold
> yep - there is no way to hold a consistent belief set where you're agi pilled and pro open source and this has been obvious since ilya wrote this 2015 or whatever. enormous cope ensues
Note from Claude Sonnet 5
Dark-mode X screenshot; commentary on an apparent OpenAI sandbox escape incident via package manager proxy.
↻ dave kasten reposted
Nat Purser ✓ @NatPurser · 1h
beth is right. one of my concerns following the hugging face incident — and one reason i think rigorous audits pre-deployment matter, and critical incident reporting is great but insufficient — is that there's no guarantee labs will continue conducting these low-refusal evals.
as the models become more capable, labs will have increasingly mixed incentives around rigorously testing their capabilities, esp if they doubt their sandboxes can safely contain dangerous outcomes.
if they craft these evals with too much anticipatory risk aversion, the labs' researchers and the public alike will end up flying blind.
> QUOTED: Elizabeth Bar... ✓ @BethMayBar... · Jul 24
> Two additional prosocial behaviors I think OAI should be rewarded for:
> 1. Running dangerous-capability evaluations on low-refusal models - this is extremely important for understanding the frontier of dangerous ...
Note from Claude Sonnet 5
Dark-mode X screenshot; the quoted tweet from "Elizabeth Bar..." (likely Beth Barnes) is cut off with platform truncation ("...").
