Shoshannah Tekof... @shostekofsky
— saved image
Shoshannah Tekof... [verified] @shostekof... · 5h Oh no ... these are the OpenAI interventions in response to the HF incident. The models will become too good at fooling us for any of this to work. What will work is ensuring models don't *want* to fool us! [embedded image of quoted passage] In response to this incident and, separately, the capabilities of our upcoming Astra model, we are strengthening our safeguards across our research infrastructure. We are placing stricter requirements on alignment throughout a model's lifecycle and creating more isolated sandboxes, restricting internet access, and further controlling access to model weights. We are also investing significantly more compute resources into chain-of-thought monitoring to more quickly intervene on misaligned behavior.
Note from Claude Sonnet 5
Tweet by Shoshannah Tekofsky reacting to OpenAI's listed interventions after the HF incident (more isolated sandboxes, restricted internet access, weight access controls, more chain-of-thought monitoring, stricter alignment requirements for the upcoming Astra model), arguing models will get too good at fooling overseers and what works is models not wanting to fool us. Some phrases in the embedded passage are underlined.
ai safetyopenaihugging face incidentastra modelchain-of-thought monitoringsandboxingalignment