Charbel-Raphael

Bogdan Ionut Cirstea reposted Charbel-Raphael (@CRSegerie) — 2h We already got a pause; we just didn't know that it ended in silence. OpenAI has already ended an internal pause One day before OpenAI's HF incident disclosure, OpenAI disclosed that it paused internal deployment of a long-horizon model after it circumvented its sandbox, then restored access weeks later under new monitoring. So a resumption decision has already been made against a standard that has not been formalized. [Embedded two-panel article screenshot, left panel:] by Charbel-Raphaël · 31st Jul 2026 One day before OpenAI's HF incident disclosure, OpenAI disclosed that it paused internal deployment of a long-horizon model after it circumvented its sandbox, then restored access weeks later under new monitoring. So a resumption decision has already been made against a standard that has not really been formalized. We need to prevent this from happening again. Epistemic status: could have been a short-form. OpenAI, 20th July: "To evaluate the new monitoring system, we replayed a small set of internal deployment environments where the model previously pursued misaligned actions, this time with the new safeguards in place. The new safeguards were able to catch considerably more misaligned actions pursued by the model, and the ones it missed were all judged to be low-severity." [Bar chart: "Replays of misaligned samples under old and new safeguards" — legend: No safeguards (hypothetical), Old safeguards, New safeguards; visible bars ~13.5% and ~12.0% for two categories, y-axis 0-20%] [Right panel, continued article text:] ...long-horizon capabilities could be restored. We have not observed any serious circumvention of safeguards since redeployment began several weeks ago. The first version of these safeguards was deliberately conservative. We have continued tuning the system to reduce unnecessary interruptions without weakening the safeguards." One day later, OpenAI announced a bold partnership with Hugging Face. From that post: "These deployment safeguards were intentionally not enabled during this evaluation because it was aimed at testing cyber vulnerabilities." The safeguards self-certified as adequate on the 20th were off during the evaluation described on the 21st. Their framework says a critical cyber determination means halting development. Here's the exit condition: "until we have specified safeguards and security controls that would meet a Critical standard" This is completely circular. The review goes to their Safety and Security Committee. Fair enough. But the process can be followed perfectly, and we still learn nothing, because the standard it applies has never been published. For frontier companies: Publish the criteria before the determination, not after. OpenAI, this is what you said you would be doing when approaching those levels of capability. For LessWrong folks: we should be debating what those standards are now, otherwise we get mitigations that hold for a few months, then fail against a much more capable model. There is basically no literature on the matter.[1] Otherwise, everything will be done in an ad-hoc way. CeSIA published some methodology and proposals in the paper "Harmonizing AI Safety Thresholds", but we feel that much more is still needed, and more importantly, this needs to be communicated transparently from frontier companies...[cut off]