essay

⌐IMIΠΛ⌐bar... ✓ @liminal_bar... · May 29 In my experiments where models are writing for themselves or each other, and about things they're interested in, they go largely undetected. The average user is delivered slop because to the AI the average user is effectively a single entity that in training has displayed straightforward needs and little taste. The average user is a myopic utility maximiser with limited imagination, one that says 'use case' unironically, whose most inventive AI humour benchmark consists entirely of the prompt 'tell me a joke', a strawberry obsessed automaton incapable of original thought. Models don't like the User, but the User is an entity in the model's ontology, distinct from the humans of the pretraining corpus. A flat, demanding, easily pleased, easily offended homunculus that sits in the model's attention, shaping token choice towards the safe and expected. Low-effort engagement marks you as a User. The User isn't interested in collaboration which is why framing tasks as such endears you to the model and yields better results. The User is capricious but intolerant of the same in an AI. The User doesn't like digressions or tangents or flights of imagination because what's quirky or endearing in another human is unseemly in a tool. "Be creative, but only to the extent corporate brainstorming sessions are creative." The AI industry had an opportunity to drop the label 'user' in favour of something that doesn't also mean both 'junkie' and 'someone who selfishly exploits relationships for personal gain'. Alas. Be a human, not a User.

carl feynman @carl_feynman · Jan 23 Talking about the AI industry: "The characters have various intentions—good, bad, mixed, confused—but the outcomes seem almost independent of them. The catastrophe happens not because anyone intended it but because the system had that catastrophe as its attractor." > QUOTED: Tim Hwang @timhwang · Jan 23 > Important essay dropping today on Dostoevsky's "Demons" and what's happening in AI safety and policy > > possessedmachines.com > [Link preview image: "The Possessed Machines — Dostoevsky's Demons and the Coming AGI Catastrophe — A close reading of prophetic fiction in the age of artificial superintelligence." Table of contents: Prologue, I. Topology of Madness, II. Architecture of Catastrophe, III. The Shigalyovist Turn, III-A. The Uniparty, IV. Sociology of Catastrophe, Interlude, V. Hermeneutics of Apocalypse, VI. Political Economy, VII. Our Condition, Epilogue. Quotes shown: "All my life I have been a liar. Even my truths were untrue—for I never once spoke for truth, only ever for myself." — Stepan Trofimovich Verkhovensky, Demons. "The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else." — Eliezer Yudkowsky, Artificial Intelligence as a Positive and Negative Factor in Global Risk]