← All topics

frontier model evaluation

2 captures, most recent first.

Peter Wildeford @peterwildeford

— saved image

Peter Wildeford... @peterwildef... · 3h
"We used a system of coordinating agents powered by our internal model. The agents had access to tools such as the ability to read from a cached version of the internet and the ability to run code. Agents were subdivided into groups with the ability to communicate within the group"

"The groups varied in size, and the group that produced the Navier–Stokes resolution involved on the order of 10,000 concurrent agents."

"At all times we maintained the same strict safeguards that we apply to all our frontier model evaluations, including monitoring and isolation."
Note from Claude Sonnet 5

Tweet quoting three passages from a lab's report on how a Navier-Stokes resolution was produced: a system of coordinating agents on an internal model with cached-internet and code-execution tools, grouped agents (about 10,000 concurrent in the successful group), and monitoring and isolation safeguards. The quoted lab is not named in the visible text.

navier-stokesmulti-agent systemsagent swarmssafeguardsfrontier model evaluationmath

Peter Wildeford @peterwildeford

— saved image

Peter Wildeford... [verified] @peterwilde... · 18h
"METR probably has more capacity to measure and evaluate frontier models than the entire US civilian government. I can't speak about the intelligence community. But the state capacity, we don't really have enough to forecast AI capabilities, build safeguards."

[quoted post]
MTS [verified] @MTSlive · Aug 26
IAPS @sevdeawesome says METR has more capacity to evaluate frontier AI models than the entire US civilian government:

"Low regret things are just increasing visibility. ...

[embedded video thumbnail: two-person podcast split screen, labelled THEO JAFFEE and SEVERIN FIELD, banner "VISITING FELLOW AT INSTITUTE FOR AI POLICY AND STRATEGY", timestamp 2:46, sponsor strip: AdQuick, databricks, ElevenLabs, MongoDB, Lovable, Are[cut off]]
Note from Claude Sonnet 5

Quote-tweet by Peter Wildeford of an MTS clip in which IAPS visiting fellow Severin Field says METR has more capacity to evaluate frontier models than the entire US civilian government. The video thumbnail shows two podcast hosts in headphones.

metrai policystate capacityfrontier model evaluationiaps