← All topics

ai auditing

3 captures, most recent first.

Michiel Bakker @bakkermichiel

— quoting @AVERIorg — saved image

I'm personally very bullish on how important technology like this could become. In the current situation, people inside a frontier lab could encode "secret loyalties" or other hidden behaviors into a model . For example, behaviors that subtly favor its developer or behaving well only when it recognizes that it is being evaluated. This may seem far-fetched today but, as labs automate more of their own workforce, it could be done by a handful of people and become much harder for the outside world to notice. Secret evals like these make such behavior harder to hide.

Over time, this kind of infrastructure could support much more comprehensive oversight across the training and deployment process. Auditing agents with access to all the training and deployment logs, could function as probes for hidden objectives. This could move us from occasional post hoc evaluations like this towards more continuous oversight, without requiring labs to expose their weights, data, or IP.

Congrats @iamtrask , @SolomonMg, @KLdivergence, @Miles_Brundage and everyone involved!

[quoted post]
AVERI [verified] @AVERIorg · 3h

Today we're announcing a historic milestone:

The first ever double-blind evaluation of a proprietary language model. This was made possible by a unique collaboration between AVERI, @GoogleDeepMind, ... [cut off]

Last edited 9:04 AM · Aug 27, 2026 · 1,264 Views
Note from Claude Sonnet 5

Second screenshot of the same Michiel Bakker thread as the previous capture: argues secure-enclave evals could expose secret loyalties or evaluation-aware hidden behaviors inserted by a few lab insiders, and could grow into continuous oversight via auditing agents without exposing weights or IP; quotes AVERI's announcement of the first double-blind evaluation of a proprietary language model. A floating compose button overlaps the quoted post.

ai evaluationsecret loyaltieshidden objectivessecure enclavecontinuous oversightaveriai auditing

Caleb Parikh @caleb_parikh

— quoting @_lamaahmad — saved image

Caleb Parikh  @caleb_parikh · 9h
It's imo extremely underrated that single digit people
(like Lama), with hard work and grit, can significantly
enable critical safety work at OpenAI.

[Quoted post]
Lama Ahmad لمى احمد  @_lamaahmad · 20h
We believe meaningful transparency requires
more than publishing our own account. It also
means giving credible external experts the
access needed to examine the evidence,
challenge our understanding, and reach their ... [cut off]
Note from Claude Sonnet 5

Quote-tweet by Caleb Parikh praising how a handful of people such as Lama Ahmad can enable critical safety work at OpenAI, quoting Lama Ahmad's truncated post about transparency requiring credible external experts to have access to examine evidence.

openaisafety worktransparencyexternal accessai auditing

@prpaskov

— saved image

Patricia Paskov  [verified badge]  @prpaskov · 49m
A well-functioning frontier AI auditing ecosystem
requires deep access with confidentiality
guarantees -- guarantees to frontier developers that
their model weights are protected, and guarantees
to evaluators that their evaluation methodologies
are not shared, stored, or trained against.

Excited to announce @AVERIorg's first pilot results,
in partnership with @GoogleDeepMind
@openminedorg and @MLCommons: a working
demonstration of how secure enclaves can meet
this constraint, and what they could unlock for a
vibrant, robust auditing ecosystem. Blog and
technical report below.
Note from Claude Sonnet 5

Tweet from Patricia Paskov announcing AVERI's first pilot results with Google DeepMind, OpenMined and MLCommons on secure enclaves for confidential access in third-party frontier AI auditing.

ai auditingaverisecure enclavesthird-party evaluationmodel weights security