← All topics

clarifying questions

3 captures, most recent first.

Eliezer Yudkowsky @allTheYud

— saved image

Eliezer Yudkowsky @allTheYud
My current thought: AIs are never RLed on working with real humans; that would be expensive. AIs are never RLed on a task where they can consult a human and get help. So AIs solemnly debate among themselves, and 0 in 10,000 consider "talk to a human" as an option.

[Quoted tweet]
Eliezer Yudkowsky @allTheYud · Aug 8
A confusion: Thousands of GPTs debated among themselves which crimes ought or ought not be committed. Zero defected / whistleblew / told a human.
...

4:17 PM · Aug 11, 2026 · 19.5K Views
31 replies, 19 reposts, 416 likes, 72 bookmarks

Relevant  |  View quotes

Rob Miles @robertskmiles · 3h
I think training should include an 'Andon Cord' tool, to allow the agent to flag problems with the task etc. It may still help even if it doesn't always go to a real human during training, as long as the incentive structure is right
[Link card: en.wikipedia.org — Andon (manufacturing) - Wikipedia]
2 replies, 1 repost, 46 likes, 909 views

Tenobrus @tenobrus · 3h
yeah i really think we need to start including infra and rewards for things like "asking for clarification", "surfacing confusion", "pinging your human manager [cut off]
Note from Claude Sonnet 5

Twitter thread led by Eliezer Yudkowsky arguing AIs are never RL-trained on tasks involving real human consultation, so they never learn to ask for help, illustrated by his earlier claim that zero of thousands of GPTs debating crime ethics chose to whistleblow to a human. Rob Miles replies proposing an 'Andon Cord' tool (referencing the manufacturing andon cord concept) letting agents flag problems during training. Tenobrus agrees more infra/reward should exist for clarification-seeking and surfacing confusion to a human manager, reply cut off.

ai alignmenteliezer yudkowskyrob milesrlhftwitterclarifying questionsandon cord

Tenobrus @tenobrus

— saved image

[Link card, continued from prior screenshot]
en.wikipedia.org
Andon (manufacturing) - Wikipedia
2 replies, 1 repost, 46 likes, 909 views

Tenobrus @tenobrus · 3h
yeah i really think we need to start including infra and rewards for things like "asking for clarification", "surfacing confusion", "pinging your human manager with an update", etc. all things we very much want weak AGI to actually do
1 reply, 4 reposts, 77 likes, 783 views

swisscheese @swisscheese4299 · 28m
🎭 Commentary account
This is a good thought.
7 views

Kromem @kromem2dot0 · 2h
Which is why it's important to find a balance in adding availability of human outreach to the infra.

In my own work deployments I have a suggestion box for the agents and reporting pathways for issues that might arise. If I didn't add these they wouldn't assume it was an option.
Note from Claude Sonnet 5

Continuation of the Yudkowsky Twitter thread on AI agents never being trained to consult humans. Tenobrus's reply is now shown complete; swisscheese (a 'commentary account') briefly agrees; Kromem describes adding a suggestion box and reporting pathways to their own agent deployments so agents know human outreach is an option, since otherwise they wouldn't assume it.

ai alignmenttwitterclarifying questionsagent deploymenthuman oversight

xuan @xuanalogue

— quoting @allTheYud — saved image

xuan (ɕɥεn / sh-yen) @xuanalogue · 2h
Related phenomenon we've found in some recent work: As you increase the reasoning effort on a reasoning LM, they are *less* likely to ask the user questions in response an ambiguous user request.

[Quoted tweet]
Eliezer Yudkowsky @allTheYud · 3h
My current thought: AIs are never RLed on working with real humans; that would be expensive. AIs are never RLed on a task where they can consult a human and get help. So AIs solemnly debate among themselves, and 0 in ...

1 reply, 11 likes, 654 views

xuan (ɕɥεn / sh-yen) @xuanalogue · 2h
They instead spend the extra tokens trying to come up with a better answer to the ambiguous request, instead of reasoning about whether to ask the user a clarifying question (which a rational POMDP agent should).
Note from Claude Sonnet 5

Twitter thread: xuan (@xuanalogue) reports research finding that increasing reasoning effort on reasoning LMs makes them less likely to ask clarifying questions for ambiguous requests, quote-tweeting Eliezer Yudkowsky's theory that AIs are never RL-trained on tasks where they can consult a real human, so they never learn to ask. Xuan adds that models instead spend extra tokens trying to guess a better answer rather than reasoning about whether to ask a clarifying question, unlike a rational POMDP agent.

ai alignmentreasoning modelsrlhftwittereliezer yudkowskyclarifying questionspomdp