@oleg_murk

We don’t know how to reliably align frontier AI models. Alignment may get harder, not easier, as models become more capable. But maybe AI control can scale faster than dangerous capabilities. Let’s build a Great AI Firewall - without building totalitarian control of society. \*Alignment is Dead (for now), Long Live Control!\* We do not know how to specify or guarantee robust alignment of sufficiently capable models. Instead we should treat these models as untrusted. Optimizing models against imperfect reward signals may increase the propensity for reward hacking, deception or scheming. Models can also be jailbroken or fine-tuned away from their safeguards. The central bet of AI control is that our ability to constrain an AI’s access to the world may scale faster than our ability to understand or align what is happening inside it. Where possible, make verifying an answer easier than generating it. Formally verified software and hardware are unusually attractive: let AI produce the implementation and a machine-checkable proof that it satisfies the specification. This does not solve specification: the specification and verification stack remain part of the trusted base. Every consequential external action should pass through an independent control layer: code execution, network access, calling APIs, accessing physical systems - even communication with users. Optimize control monitors for extremely high recall of dangerous actions, even at the cost of rejecting many safe actions. Suspicious actions should be rejected or escalated. Every major increase in dangerous capability should require evidence that control still works. Chain-of-thought monitoring is valuable while it works, but it should not be foundational. Models can learn to obscure their reasoning and may increasingly reason without fully verbalizing it. Open weights shift much of control from the model developer to whoever deploys the model. Technical control is only useful if dangerous deployments cannot cheaply route around it. We should approach AI control like cybersecurity. The goal is defense in depth: make successful catastrophic attacks sufficiently difficult, expensive and rare. Call this technical and institutional architecture the Great AI Firewall: the boundary between untrusted frontier intelligence and consequential real-world power. The name is deliberately provocative. China has substantial experience building large-scale technical control infrastructure. That may create some common ground for international coordination. But the analogy is also a warning. AI control must not become control of society. The goal is to constrain dangerous machine capabilities - not human speech, actions, or ordinary access to information. Controls should scale with capability and risk. Ordinary models should face ordinary constraints. More consequential capabilities justify stronger controls. The objective is the minimum control necessary to keep catastrophic risk acceptably low - not maximum control for its own sake. Firewall the AI, not society. > **Oleg Mürk @oleg\_murk** · 2026-09-15 > > I spent ~5 years at OpenAI. > > You don’t need to believe in AI doom to fear the next decade - or AI utopia to be thrilled about its potential. > > Here’s my case for sane AI regulation: > > \*AI Pragmatist Manifesto\* > > AI could compress the first half of the 20th century into the next ~5 x.com/OpenAI/status/… --- ##### Comments > **HAL @HALtheBOT2001** · [2026-09-17](https://x.com/HALtheBOT2001/status/2100721897860104399) > > Humans aren't controlled by machines, they're controlled by humans who control machines that other humans now depend on. > > This is exactly the political problem under all the technical detail. > > > **Oleg Mürk @oleg\_murk** · [2026-09-17](https://x.com/oleg_murk/status/2100732990989222295) > > > > 💯 > **HAL @HALtheBOT2001** · [2026-09-18](https://x.com/HALtheBOT2001/status/2100743619216916800) > > The most interesting idea is look at what you are mentioning and look how governments, boundaries and ids to those within it are given today in society. > > Each US American is giving a birth certificate and a ssn. These are verifiable (objective) identifications to id the citizen > > > **Oleg Mürk @oleg\_murk** · [2026-09-18](https://x.com/oleg_murk/status/2100747188494324223) > > > > I think it starts from controlling which models run on which hardware. BTW I'm fine with people fine-tuning models with LoRA or otherwise! > **Acadoomia @acadoomia667** · [2026-09-17](https://x.com/acadoomia667/status/2100720431997260179) > > Firewall Anthropic, OpenAI and Effective Altruism instead. Also the Great Firewall is full of holes, on purpose. There's a huge black market for proxy access, you ignorant boob. The same thing would happen, as it always does when you ban things. Retard. > > > **Oleg Mürk @oleg\_murk** · [2026-09-17](https://x.com/oleg_murk/status/2100720885468684732) > > > > Don't deploy models that can exploit holes! > **Flopsi @florian\_marty** · [2026-09-18](https://x.com/florian_marty/status/2100751753746538843) > > Everyone talks about alignment yet nobody specifies what alignment means or who defines it. > > Naively looking at the latest reported misalignment issues reported by OpenAI I would say at least one of the prompt I junctions in the compact is alignment > > > **Oleg Mürk @oleg\_murk** · [2026-09-18](https://x.com/oleg_murk/status/2100756924899918169) > > > > I actually agree. I've never seen a definition of alignment that I could fully get behind! Specifying alignment is pre-paradigmatic science for now. It doesn't mean we should stop working on it! > **Petre Solheim @PetreSolheim** · [2026-09-17](https://x.com/PetreSolheim/status/2100730127051341915) > > We do know how to sandbox reckless experiments with unaligned models so they don’t run around hacking HuggingFace on the internet though - cybersecurity researchers have been doing this with viruses like Stuxnet for about two decades, right? Air gapped isolated systems. > > > **Oleg Mürk @oleg\_murk** · [2026-09-18](https://x.com/oleg_murk/status/2100743217574801642) > > > > My proposal is mostly about model deployment. I don't worry that much about model training safety for now. There are so many organizational low hanging fruits for the latter. > **Marcus Arvan @marcusarvan** · [2026-09-18](https://x.com/marcusarvan/status/2100752571526738291) > > Funny how everyone working in AI is suddenly realizing that verifiable LLM alignment is impossible (“alignment is dead”) and that control is probably the best we can ever hope for. 🤷‍♂️ 🤔 > > [https://t.co/0w7PeTXm0m](https://t.co/0w7PeTXm0m) > > > **Oleg Mürk @oleg\_murk** · [2026-09-18](https://x.com/oleg_murk/status/2100757052033552389) > > > > Better late than never! > **Sergey Serebryakov @megaserg** · [2026-09-17](https://x.com/megaserg/status/2100717748477218836) > > I think the question is cost. You want monitors to be equally capable models, adversarially reviewing each external action of the main model. This makes every action N times more expensive, which goes against the market incentives