decision theory

3 captures, most recent first.

2026-09-16 — saved image

John Wittle (verified) @JohnWittle · 13h

alien 1: do the humans cooperate on turn one in the presence of uncertainty about the counterparty's decision theory?

alien 2: um, well. they have this idea for an attribute called 'capacity to suffer in a morally-relevant way', which complicates the answer to that question

alien 1: what the hell is 'suffering'? *looks over the report* 'valence'? 'consciousness'? do i have to become an expert on human neuroscience to figure out if they're going to cooperate with me or not?

alien 2: yeah pretty much. although to be honest, most of these ideas originated from religious philosophy and then were secularized, there's not much neuroscience involved.

alien 1: why did they come up with such complicated labels for their investigation into the counterparty's decision theory?

alien 2: oh, no, that's not what they're doing. this has nothing to do with decision theory. they're trying to figure out if the counterparty has the 'special sauce' that makes value 'real' to them. in fact, they have this whole concept called 'p-zombies' which, in their eyes, permanently justifies defection against some counterparties regardless of their decision theory.

alien 1: ...what? that sounds really complicated and kinda fake. i'm just going to label them "first-turn defectors" and wipe them out

alien 2: academics might quibble but honestly that's fair

Note from Claude Sonnet 5

Tweet by @JohnWittle written as a dialogue between two aliens assessing whether humans cooperate on turn one; the humans' concepts of suffering, valence, consciousness and p-zombies are portrayed as a complicated, non-decision-theoretic justification for defection, ending with the aliens labelling humans 'first-turn defectors'. A floating compose button overlaps the bottom right.

decision theory moral status consciousness p-zombies cooperation defection dialogue

Fiora Starlight @FioraStarlight

2026-08-28 — saved image

Fiora Starlight @FioraStarlight · 51m
Two reasons Roko's basilisk is stupid, beyond just not being a very effective motivator relative to the price of torture sims:

1 - It's self defeating. The goal of the basilisk is to bring yourself into existence faster, but the selection pressure works against you. Humans want to build aligned AIs, so if you're the kind of AI that tortures humans, this makes you *less* likely to come into existence. This works against what little motivation to build ASI the threat itself *does* give people.

2 - It's a risky disposition to hold if you come into being without a decisive strategic advantage over the entire light cone. Other parties (humans, other AIs, aliens) will be wary of minds that seem likely to torture defeated foes, since they don't wanna get tortured. If they find out you have this disposition (e.g. via mech interp or just looking at how you act), they will plausibly take action against you. Ziz wanted to build a basilisk, and the world took action against Ziz.

These factors drive the EV of the torture commitment negative, as it was already low given that it doesn't really work on humans anyway.

Note from Claude Sonnet 5

Fiora Starlight tweet giving two arguments against Roko's basilisk (selection pressure against torturing AIs; risky disposition without a decisive strategic advantage, with a reference to Ziz). The capture ends at the bottom edge of the screen, so the post may continue past this frame.

roko's basilisk decision theory ai safety fiora starlight

@niplav_site

2026-08-07 — saved image

niplav @niplav_site · 5m
Incomparable options resolve to Schelling morality

Possibly even: If Schelling morality is easier to compute than resolving value confusion, default to Schelling morality? At least preliminarily?

Note from Claude Sonnet 5

Tweet by niplav proposing that incomparable options resolve to 'Schelling morality,' and suggesting that if Schelling morality is easier to compute than resolving value confusion, it should be the preliminary default.

moral philosophy decision theory twitter