Miles Brundage @Miles_Brundage
— saved image
Miles Brundage @Miles_Brundage · 1h It was an honor and a ton of fun to work on this unique collaboration between AVERI, Google DeepMind, OpenMined, and MLCommons. Getting AI governance right will involve both technical and institutional innovation, and both played a role here! [quoted post] AVERI @AVERIorg · 1h Today we're announcing a historic milestone: The first ever double-blind evaluation of a proprietary language model. This was made possible by a unique collaboration between ... [2 replies, 7 reposts, 34 likes, 1.6K views] Miles Brundage @Miles_Brundage This diagram summarizes the different roles played by each party. See our blog post and the technical report for a lot more detail! [attached diagram, cut off at the bottom of the screenshot; visible portion] GOOGLE DEEPMIND (lock icon) Proprietary model weights Encrypted by Google DeepMind (arrow down) MLCOMMONS (lock icon) Never-before-seen benchmark prompts Encrypted by AVERI (arrow down) SECURE ENCLAVE ENGINEERED BY OPENMINED [further contents of the diagram cut off]
Note from Claude Sonnet 5
Miles Brundage announces a collaboration between AVERI, Google DeepMind, OpenMined and MLCommons that produced the first double-blind evaluation of a proprietary language model, with a diagram of the roles: DeepMind's encrypted model weights and MLCommons' never-before-seen benchmark prompts (encrypted by AVERI) flowing into a secure enclave engineered by OpenMined. The diagram is cut off at the bottom.
ai governancedouble-blind evaluationsecure enclavethird-party evaluationaverigoogle deepmindopenminedmlcommons