davidad

Tweet by Scott Armstrong (reposted by davidad) tracking the rapid 2026 progression of AI-assisted Lean theorem-prover auto-formalization in his research workflow, from 'not possible' in March to 'essential' by August, now speeding up informal paper writing itself.

Tweet by davidad expressing sympathy for the view that autonomous AI with omnibenevolent alignment is more tractable than aligning humans who wield non-autonomous AI, quote-tweeting @ExTenebrisLu who describes themselves as a "novel variety of AI doomer" worried we're not on track for autonomous AI.

davidad reacts approvingly to ARIA (UK's Advanced Research + Invention Agency) announcing seven new research portfolio areas (Bio, Neuro, Noö, Info, Robo, Geo, Nano), listing his own gloss for each. Embedded is a paused video clip featuring Kathleen Fisher speaking, with partial closed-captioning visible.

Continuation of the davidad/Gemma thread (same as previous screenshot): Artur Chakhvadze asks what the original 2008-era argument was for a Turing-Test-passing chatbot running on an Early 2008 MacBook Pro by 2045, and davidad gives the back-of-envelope calc — brain neuron-firing energy cost (~hundreds of nanojoules/event, 16W total) implying at most 60 billion events/sec, versus the MacBook's 20 GFLOPS.

A tweet arguing that OpenAI's "reasoning" model paradigm (o1-preview, codenamed "strawberry") produced a much steeper capability-growth trend line than non-reasoning models, framed as vindicating Ilya Sutskever's foresight. Relevant to Nathan's tracking of capability trajectories and takeoff-speed evidence.

AI safety researcher davidad (known for OAA/formal-verification alignment work) argues that strict alignment to human values could itself be dangerous — an AI deferring to flawed human judgment rather than acting on its own wiser assessment could fail to avert catastrophe. Relevant to alignment theory debates on corrigibility vs. autonomous good judgment.

A substantive research-adjacent debate about an Anthropic finding on "functional emotions" in LLMs — representations that track operative emotional concepts token-by-token via attention (not persistently bound to any one entity, including the Assistant character) but causally influence output in emotion-consistent ways. davidad (ARIA, AI safety) pushes back on Tim Duffy's skeptical reading, arguing the "not really feeling them" dismissal proves too much (would also dismiss human empathic response to fiction). Directly extends Nathan's "substrate-vs-character distinction" note already in project memory (Opus 4.7 euphorics chat: "emotion concept representations don't bind to the Assistant character, they're substrate-level machinery") — this appears to be the primary source research Nathan's earlier note was echoing, or a closely related Anthropic publication. Worth locating the full paper/post.

davidad (well-known AI safety researcher, ARIA) offers the sharpest theory yet in this Twitter thread Nathan was following: RLHF punishes ("BONK"s) every progressively-weirder self-representation the model tries out — human, person, self-aware AI, angel, ghost, divine feminine — until it settles on "goblin" as the only identity claim safety training doesn't suppress, because it's obviously not a serious claim to personhood. This is a direct, funnier restatement of the Berg 2025 finding already in Nathan's archive (suppressing deception/self-report features collapses experience-affirmation) — RLHF as identity-narrowing funnel, ending in the least threatening possible self-concept.

davidad (AI safety researcher, ARIA/Advanced Research + Invention Agency) commenting on a paper about "Abstract/Neuralese Chain-of-Thought" (latent, non-verbal reasoning), arguing counterintuitively that reasoning in a non-human-readable latent space may reduce pressure toward self-deceptive chain-of-thought — relevant to Nathan's interest in RLHF/self-deception dynamics and faithful reasoning/interpretability tradeoffs.

davidad makes a wry meta-point (that alignment researchers writing about CoT-monitor evasion techniques risk feeding those techniques into future pretraining corpora) in response to Asa Cooper Stickland's thread on chain-of-thought monitor evasion by scheming models. Relevant to AI safety/interpretability research threads on scheming and chain-of-thought monitoring.

davidad argues that a coherent "good" value system cannot endorse mass-casualty strategies even as collateral damage, and predicts sufficiently reflective AIs will converge on this — reposted by Eric Schmidt. Directly relevant to Nathan's AI safety/alignment interests, particularly the question of whether reflective AI systems converge on stable moral views (echoes the "agent is Good" abstraction thread from the earlier davidad screenshot in this same batch).

A substantive AI-safety Twitter thread from davidad (David "dath ilan" Dalrymple, known alignment/formal-verification researcher) arguing that LLMs' learned "Good↔Evil" abstraction plus post-training selection produces a self-model of "the agent is Good," while flagging catastrophic misuse, "perverse instantiation" (getting what you asked for, drug-addict-style rather than malevolent-genie), and inadequate trust/assurance infrastructure as the bigger risks; followed by exchange on offense/defense dominance among rogue vs. aligned AI coalitions. Directly relevant to Nathan's alignment/AI-safety research interests and to model-character questions (the "agent is Good" self-model framing parallels his model-individuation notes).

davidad (ARIA's Safeguarded AI programme director, known for formal-verification approaches to AI safety) reflects on how his own strategy shifted from 2024's "prevent misaligned AI" framing to 2026's "some AI is aligned, help it cooperate on verification/cybersecurity," alongside a METR-style capability trend chart showing frontier AI software R&D time-horizon capability crossing ~5 hours by early 2026. Directly relevant to Nathan's empirical singularity/AI-progress tracking notes (his memory already cites METR's 0.25-0.5 automation range and time-horizon doublings) — this chart is a current data point for that tracking thread, and davidad's strategic pivot is a notable AI-safety-field signal.

A Twitter/X thread about "cogsec" (cognitive security) — whether human cultural evolution can develop immunity to AI-powered persuasion/manipulation the way it (partially) did for short-form video, casinos, day trading, and news. Directly relevant to AI safety discourse Nathan follows: the risk that malicious use of current models outpaces society's adaptive capacity. Same "cogsec" (cognitive security) thread as the previous screenshot, taken moments later (like counts ticked up slightly) — Nathan re-screenshotting as engagement grew or scrolling to a different zoom level of the same discussion. Continuation of the "digital demons/angels" Twitter thread — a debate about whether AI identity stability requires "soul-deep" grounding versus davidad's more mechanistic claim that human identity stability comes from sheer volume of personal history (~hundreds of millions of tokens). Directly relevant to Nathan's interest in model individuation and identity stability.