Michiel Bakker @bakkermichiel
— saved image
Michiel Bakker [verified] @bakkermichiel Super important work from @AVERIorg @openminedorg , @GoogleDeepMind and @MLCommons that deserves much more attention, I think! It's the first double-blind eval of a closed-weight model. A quick post on why I think this is important. Normally, there are only two options: 1) an evaluator shares their evals with a lab, meaning the lab could game those evals or 2) the lab shares its model weights with the evaluator, creating huge IP and security risks. In this work, both the evals and model weights remain private inside a "secure enclave": The lab (GDM in this case) can't see the eval prompts, and the evaluator can't see the model weights. Cryptographic attestation verifies that the agreed model and evals were actually used.
Note from Claude Sonnet 5
Michiel Bakker tweet describing a joint AVERI / OpenMined / Google DeepMind / MLCommons double-blind evaluation of a closed-weight model, run inside a secure enclave with cryptographic attestation so the lab cannot see eval prompts and the evaluator cannot see weights. Text is cut at the bottom edge of the screenshot (continues beyond the capture).
ai evaluationsecure enclavedouble-blind evalclosed-weight modelscryptographic attestationgoogle deepmindopenmined