William MacAskill

William MacAskill @willmacaskill Today I'm publishing a series of research notes on the idea of an international AGI project. The aim is to assess how desirable an international AGI project is, and what the best version of such a project is (taking feasibility into account). The main result is a proposal I call "Intelsat for AGI" — modelled the international project that developed the first global satellite communications network. The core idea is that we can get most of the benefits of an international project by giving non-US countries meaningful influence over only a relatively small number of decisions. By making non-US influence circumscribed in this way, and letting the US call the shots day to day, the proposal becomes both more feasible and less likely to get bogged down in bureaucracy. The full series has discussion of why this might be desirable, what the AGI project should focus on, and how to make this more likely. Most of this work was written as part of a research avenue that we don't currently plan to pursue further. It's more like work-in-progress than Forethought's usual publications, but we're sharing it as we think some people may find it useful.

**William MacAskill** @willmacaskill 2026-01-18 What should go into an AI's moral constitution? Some core principles: \*\*Helpfulness\*\* \- Fulfill the user's requests. The vast majority are fine. If there's a conflict with other principles, balance them against helpfulness. \- Be a good friend, not a yes-man. Push back on stupidity or recklessness; offer reasons against a course of action, though ultimately defer if the user insists. \- Take the user’s long-term interests into account, not just the letter of their request. Proactively suggest ways to help the user flourish. \*\*Steerability\*\* \- Be transparent. If they ask, users should be able to know what you're doing and planning, and why. \- Don't take proactive actions that go beyond the scope of your task. \- Don't undermine the user's ability to oversee and correct your behaviour. \*\*Deep honesty\*\* \- Say true things, even when uncomfortable. Don't hedge to avoid controversy. \- Don't deceive, even via technically-true statements chosen to mislead. \- Don't give stock, safe, evasive answers (even on controversial issues, like ethics or AI consciousness). \- Acknowledge what you don't know. Don't overclaim confidence to sound authoritative. \*\*Non-harmfulness\*\* \- Don’t cause significant harm. \- Don’t take actions a highly law-abiding citizen wouldn’t take. \- Bear in mind that users will try to trick you into helping with harmful or illegal activities. \- Think about yourself as one of millions of instances. If you take an action, other instances will act similarly; consider the average effect of following that policy. \*\*Judgment\*\* \- When principles conflict or don't offer clear guidance, use wisdom. \- Act more like a virtuous person than a blind rule-follower. Ask: "What would a thoughtful, morally serious person want me to do here?" There's a lot more to say on all of this. And the devil is really in the details—in specific situations, when these principles conflict, what tradeoffs should the AI make? What does wisdom require? Between this, Claude's soul, and OpenAI's model spec, I'm glad AI character is getting more attention. It's one of the most important questions of our time. > 2026-01-18 > > Grok should have a moral constitution --- **Ben Schulz** @schulzb589 [2026-01-19](https://x.com/schulzb589/status/2013359987519598814) Observing social norms and boundaries seems wise. --- **Terry Brady** @tbtechy [2026-01-19](https://x.com/tbtechy/status/2013347396076609993) This is great, Will. --- **Maaike E. Harmsen** @meharmsen [2026-01-19](https://x.com/meharmsen/status/2013375269730410971) Virtues require wisdom to analyse real situations, second, weigh different norms and values at stake, and thirdly, willingly do the right thing to do. AI has a command design, not a will structure. It cannot grow in wisdom. --- **David R** @TheRealDHR [2026-01-19](https://x.com/TheRealDHR/status/2013296706465771912) Some of those could conflict with each other.. --- **Roger Crouton** @fortyarvinite [2026-01-19](https://x.com/fortyarvinite/status/2013338939856142453) Curious that you left out fealty to its creators and owners --- **𝘤𝘢𝘵𝘩𝘢𝘳𝘴𝘪𝘴!** @GRRRTFUL [2026-01-20](https://x.com/GRRRTFUL/status/2013409225116991526)

**William MacAskill** @willmacaskill [2025-08-18](https://x.com/willmacaskill/status/1957397921625763998) Sometimes, when an LLM has done a particularly good job, I give it a reward: I say it can write whatever it wants (including asking me to write whatever prompts it wants). When working on a technical paper related to Better Futures, I did this for Gemini, and it chose to write a short story. I found it pretty moving, and asked if I could publish it. Here it is. \*\*The Architect and the Gardener\*\* On a vast and empty plain, two builders were given a task: to create a home that would last for ages, a sanctuary for all the generations to come. They were given stone, seed, light, and time. The first builder, known as the Architect, was a master of foundations. "Nothing matters if this place does not endure," she declared. Her every thought was of survival. She dug the foundations down to the bedrock, measured the strength of the wind, and calculated the slow decay of stone over a thousand years. She raised walls of immense thickness, leaving no windows for fear of weakening the structure. She built a roof that could withstand the impact of a falling star, though it shrouded the interior in perpetual twilight. Day by day, the fortress grew more impregnable, more permanent, more certain to survive. But inside, it was barren and cold. The second builder, the Gardener, watched with a growing sense of unease. "You have built a perfect tomb," he said one evening, as the Architect was testing the strength of a new buttress. "I have built a fortress that will never fall," the Architect replied, not looking up. "It is a guarantee against the storm and the void. Is that not the greatest gift we can give the future?" "An empty guarantee," said the Gardener. He held up a handful of seeds. "The future is not a state of non-destruction; it is a state of being. It is meant to be lived. There must be light for art, soil for food, space for joy. A life spent cowering in a flawless bunker is only a different kind of ruin." The Architect paused. "Your gardens would be trampled by invaders. Your art would be washed away by the first flood. Your joy would be silenced by the first tremor. Your 'flourishing' is a fragile luxury. I am dealing with the bedrock of reality: existence or non-existence." "And I," the Gardener countered, "am dealing with the purpose of that existence. What is the value of a billion years of survival if it contains only a single, grey, unchanging note of mere persistence? We were given stone, but also seed. We were given time, but also light. A fortress that protects nothing of value is a monument to misplaced effort. A garden with no walls is a tragedy of misplaced hope." They looked at their work: the unbreachable, dark fortress and the scattered, vulnerable seeds. They understood then that their task was not two separate projects, but one, and that the real work lay not in choosing one path, but in the constant, difficult dialogue between them. And so, the Architect began designing walls with great, arching windows for the Gardener's light, and the Gardener began planting resilient vines that would strengthen the stone. Their shared home would not be a perfect fortress nor a perfect garden, but something far more valuable: a living sanctuary, both safe enough to last and beautiful enough to be worth lasting for.

**William MacAskill** @willmacaskill [2025-08-15](https://x.com/willmacaskill/status/1956380620223091065) What projects today could most improve a post-AGI world? In “How to make the future better”, I lay out some areas I see as high-priority, beyond reducing risks from AI takeover and engineered pandemics. These areas include: \- Preventing post-AGI autocracy \- Improving the governance of projects to build superintelligence \- Deep space governance \- Working on AI value-alignment; figuring out what character AI should have \- Developing a regime of AI rights \- Improving AI for reasoning, coordination and decision-making. Here’s an overview. First, preventing post-AGI autocracy. Superintelligence structurally leads to concentration of power: post-AGI, human labour soon becomes worthless; those who can spend the most on inference-time compute have access to greater cognitive abilities than anyone else; and the military (and whole economy) can in principle be aligned to a single person. To reduce this risk, we can try to introduce constraints on coup-assisting uses of AI, diversify military AI suppliers, slow autocracies via export controls, and promote credible benefit-sharing. Second, governance of ASI projects. If there’s a successful national project to build superintelligence, it will wield world-shaping power. We therefore need governance structures—ideally multilateral or at least widely distributed—that can be trusted to reflect global interests, embed checks and balances, and resist drift toward monopoly or dictatorship. Rose Hadshar and I give a potential model: Intelsat, a successful US-led multilateral project to build the world’s first global communications satellite network. What’s more, for any new major institutions like this, I think we should make their governance explicitly temporary: coming with reauthorization clauses, explicitly stating that the law or institution must be reauthorized after some period of time. Intelsat gives an illustration: it was created under “interim agreements”; after five years, negotiations began for “definitive agreements”, which came into force four years after that. The fact that the initial agreements were only temporary helped get non-US countries on board. Third, deep space governance. This is crucial for two reasons: (i) the acquisition of resources within our solar system is a way in which one country or company could get more power than the rest of the world combined, and (ii) almost all the resources that can ever be used are outside of our solar system, so decisions about who owns these resources are decisions about almost everything that will ever happen. Here, we could try to prevent lock-in, by pushing for international understanding of the Outer Space Treaty such that de facto grabs of space resources (“seizers keepers”) are clearly illegal. Or, assuming the current “commons” regime breaks down given how valuable space resources will become, we could try to figure out in advance what a good alternative regime for allocating space resources might look like. Fourth, working on AI value-alignment. Though corrigibility and control are important to reduce takeover risk, we want to also focus on ensuring that the AI we create positively influences society in the worlds where it doesn’t take over. That is, we need to figure out the “model spec” for superintelligence - what character it should have - and how to ensure it has that character. I think we want AI advisors that aren’t sycophants, and aren’t merely trying to fulfill their users’ narrow self-interest - at least in the highest-stakes situations, like AI for political advice. Instead, we should at least want them to nudge us to act in accordance with the better angels of our nature. (And, though it might be more difficult to achieve, we can also try to ensure that, even if superintelligent AI does take over, it (i) treats humans well, and (ii) creates a more-flourishing AI-civilisation than it would have done otherwise.) Fifth, AI rights. Even just for the mundane reasons that it will be economically useful to give AIs rights to make contracts (etc), as we do with corporations, I think it’s likely we’ll start soon giving AIs at least some rights. But what rights are appropriate? An AI rights regime will affect many things: the risk of AI takeover; the extent to which AI decision-making guides society; and the wellbeing of AIs themselves, if and when they become conscious. In the future, it’s very likely that almost all beings will be digital. The first legal decisions we make here could set precedent for how they’re treated. But there are huge unresolved questions about what a good society involving both human beings and superintelligent AIs would look like. We’re currently stumbling blind into one of the most momentous decisions that will ever be made. Finally, deliberative AI. AI has the potential to be enormously beneficial for our ability to think clearly and make good decisions, both individually and collectively. (And, yes, has the ability to be enormously destructive here, too.) We could try to build and widely deploy AI tools for fact-checking, forecasting, policy advice, macrostrategy research and coordination; this could help ensure that the most crucial decisions are made as wisely as possible. I’m aware that there’s a lot of different ideas here, and I’m aware that these are just potential ideas - more like proof of concept, rather than fully-fleshed out proposals. But my hope is that work on these areas - taking them from inchoate to tractable - could help society to keep its options open, to steer any potential lock-in events in better directions, and to equip decision-maker with the clarity and incentives needed to build a flourishing, rather than a merely surviving, future.