← All topics

scalable oversight

3 captures, most recent first.

Andy Hall @ahall_research

— quoting @boazbaraktcs — saved image

Andy Hall [verified] @ahall_research · 1h

Democratic accountability now requires that elected representatives of "the people" have access to AI systems that can oversee and audit other AI systems. Look at the details of the METR report and you can see why — the only plausible way to dig into the entire trail of deliberation and action is with AI.

So then we have to figure out, how does that work? How can overseers know their oversight model is honest and accurate?

[quoted post]
Boaz Barak [verified] @boazbaraktcs · 17h

Thank you to @RyanGreenblatt , @ajeya_cotra , @HjalmarWijk for this report! I assigned it as required reading for students in my AI safety course.

... [cut off]
Note from Claude Sonnet 5

Andy Hall tweet arguing democratic accountability now requires elected representatives to have AI systems that can audit other AI systems, citing the METR report, and asking how overseers can know their oversight model is honest; quotes Boaz Barak thanking the report's authors and assigning it in his AI safety course.

ai safetyai oversightdemocratic accountabilitymetr reportscalable oversight

Henry Shevlin @dioscuri

— quoting @RyanGreenblatt — saved image

Henry Shevlin [verified] @dioscuri · 2h
This is worrying. Seems like building better tools for incident analysis should be one of the absolute top priorities of safety research.

[quoted text box, no author header visible]
The difficulty of understanding incidents and overseeing AI agents appears to be growing faster than the rate at which more capable AIs help us with oversight and understanding. As in, AI capabilities (and propensities?) for achieving large, ambitious, and misaligned objectives are growing faster than our ability to understand what these agents are doing.

[quoted post]
Ryan Greenblatt [verified] @RyanGreenblatt · 9h
I was the main person doing transcript analysis for this investigation of the Hugging Face incident. My main takeaway: We don't have good approaches for understanding/overseeing the activity and aims of AI 'swarms'.... [cut off]
Note from Claude Sonnet 5

Henry Shevlin argues better incident-analysis tools should be a top safety research priority, with a quoted passage that oversight difficulty is growing faster than AI help with oversight, and quotes Ryan Greenblatt's takeaway from analyzing transcripts of the Hugging Face incident. The middle quoted box has no visible author attribution.

hugging face incidentincident analysisoversightai swarmssafety research prioritiesscalable oversight

Miles Brundage @Miles_Brundage

— saved image

Miles Brundage @Miles_Brundage · 4h
A bit concerning that a big part of the safety story from AI companies is "we'll use AIs to oversee AIs + help make sense of what they're doing" given that:

- widely deployed AIs already use confusing jargon
-expert mathematicians don't fully understand the latest AI discoveries
Note from Claude Sonnet 5

Tweet by Miles Brundage expressing concern that AI companies' safety plans rely on 'AIs overseeing AIs,' given that deployed AI already produces confusing jargon and expert mathematicians can't fully understand the latest AI discoveries.

ai safetyscalable oversightmiles brundagetwitter