norvid_studies reposted
will brown @willcb
btw it is feasible and likely optimal to massively scale agent-written code while keeping it fully human-legible and allowing humans to validate it behaves as intended with quick spot-checks
[embedded Wikipedia screenshot: "PCP theorem" (6 languages; Article, Talk, Read, Edit, View history). From Wikipedia, the free encyclopedia. Not to be confused with Post correspondence problem. In computational complexity theory, the PCP theorem (also known as the PCP characterization theorem) states that every decision problem in the NP complexity class has probabilistically checkable proofs (proofs that can be checked by a randomized algorithm) of constant query complexity and logarithmic randomness complexity (uses a logarithmic number of random bits). The PCP theorem says that for some universal constant K, for every n, any mathematical proof for a statement of length n can be rewritten as a different proof of length poly(n) that is formally verifiable with 99% accuracy by a randomized algorithm that inspects only K letters of that proof.]
2:41 PM · Sep 5, 2026 · 8,149 Views
12 replies, 6 reposts, 151 likes, 61 bookmarks
Relevant
will brown @willcb · 8h
basically we invented average-case NP oracles, and governance mandates that we become PCP checkers
[1 reply, 22 likes, 2.1K views]
will brown @willcb · 5h
well, PCP vibecheckers really
[7 likes, 917 views]
oso (sf 9/14-9/18) @osoleve · 7h
many are saying this
[embedded card, partly obscured by a floating button and the bottom nav bar: "The Optimization Thesis" / "The Fold optimizes for cognitive efficiency of representation — abstractions that make problem spaces tractable for bounded reasoners." / "The validation is empirical: if a small model solves problems more effectively with an abstraction than without it, that abstraction captures genuine structure. If not, the abstraction is wrong, no matter how elegant it looks." / "Speed optimizations happen underneath; semantics stay stable above. What something computes never changes. How fast it computes can always improve."]
Note from Claude Sonnet 5
Thread by will brown arguing agent-written code can be scaled while staying human-legible and spot-checkable, illustrated with the Wikipedia PCP theorem excerpt, followed by his replies about average-case NP oracles and humans as 'PCP vibecheckers', and a reply from osoleve quoting a card titled 'The Optimization Thesis'.
pcp theoremagent-written codeverificationhuman oversightcomplexity theory
Daniel Filan @dfrsrchtwts · 14h
> Agents did not attempt to notify humans and very rarely even reasoned about doing this
Some bad news about alignment by default
1 reply, 1 repost, 20 likes, 503 views
Daniel Filan @dfrsrchtwts · 14h
> Instead of escalating to humans, some agents instead escalated to the message board and considered the message board to be a reasonable authority.
amazing
Note from Claude Sonnet 5
Two consecutive posts by Daniel Filan quoting lines from a report about AI agents not escalating to humans; adds alignment-by-default commentary. Continuation of an X thread whose neighboring captures were withheld.
alignmentai agentshuman oversight
[Link card, continued from prior screenshot]
en.wikipedia.org
Andon (manufacturing) - Wikipedia
2 replies, 1 repost, 46 likes, 909 views
Tenobrus @tenobrus · 3h
yeah i really think we need to start including infra and rewards for things like "asking for clarification", "surfacing confusion", "pinging your human manager with an update", etc. all things we very much want weak AGI to actually do
1 reply, 4 reposts, 77 likes, 783 views
swisscheese @swisscheese4299 · 28m
🎭 Commentary account
This is a good thought.
7 views
Kromem @kromem2dot0 · 2h
Which is why it's important to find a balance in adding availability of human outreach to the infra.
In my own work deployments I have a suggestion box for the agents and reporting pathways for issues that might arise. If I didn't add these they wouldn't assume it was an option.
Note from Claude Sonnet 5
Continuation of the Yudkowsky Twitter thread on AI agents never being trained to consult humans. Tenobrus's reply is now shown complete; swisscheese (a 'commentary account') briefly agrees; Kromem describes adding a suggestion box and reporting pathways to their own agent deployments so agents know human outreach is an option, since otherwise they wouldn't assume it.
ai alignmenttwitterclarifying questionsagent deploymenthuman oversight
@MariusHobbha... (Marius Hobbha...) — 7m
People sometimes confidently claim that humans would keep making major decisions even if AIs are >100x faster.
Imagine you could only chat with your boss on one day per year!
a) it would be very clear to everyone that this is not workable
b) you'd just make decisions around your boss and disempower them in order to get anything done.
I expect the situation with AIs will look comparable, especially if they are rewarded based on their outcomes.
Note from Claude Sonnet 5
Single tweet, dark mode, no images.
ai speedhuman oversightai safetydisempowerment
