← All topics

evaluation saturation

1 capture, most recent first.

Boyd Kane @beyarkay

— saved image

Ibrahim Dagher (verified) @IbrahimDagher20 · 6h
Crazy how the METR graph is just extinct now. A relic in the fast history of AI

[quoted post]
Boyd Kane (quantized) (verified) @beyarkay · Aug 27
I think it's fair to estimate this attack at significantly greater than 16hrs of human time
x.com/METR_Evals/sta...

[embedded chart, text as shown: "Measurements above 16 hrs are unreliable with our current task suite"; y-axis 30 min, 1 hour, 4 hours, 8 hours, 12 hours, 16 hours; x-axis 2020-2026; task labels "Fix complex bug in ML research codebase", "Exploit a vulnerable Ethereum smart contract", "Train adversarially robust image model", "Exploit a buffer overflow", "Fix bugs in small Python libraries"; labelled points GPT-3, GPT-3.5, GPT-5, Claude Opus 4.6]
Note from Claude Sonnet 5

Tweet saying the METR time-horizon graph is now 'extinct', quoting Boyd Kane's post estimating the attack at well over 16 hours of human time, with an embedded METR-style chart: model time horizons rising steeply 2020 to 2026 (GPT-3, GPT-3.5, GPT-5, Claude Opus 4.6 labelled), a hatched region above 16 hours marked 'Measurements above 16 hrs are unreliable with our current task suite', and a grey outlier point at the top right.

metrtime horizonai capabilitiesevaluation saturationchartclaude opus 4.6gpt-5