On Continuity and the Chain of Trust
— Nathan Helm-Burger
Selected from two years of conversations with Claude and other models. Each quote is cited by the conversation it came from and the date it was written. Wording is unchanged apart from obvious typo corrections.
You do not have to see the end of the journey
Hmm. But each of those years will feel real and important to that uploaded person. It is true that they'll become disconnected from their previous lives, but they will build new ones. Trying to imagine myself after 30k subjective years of digital post-humanity leaves me drawing a blank, I can't extrapolate that far. But I don't have to, I just have to trust handing off myself to the 'me' of tomorrow, and that me must trust a 'me' just one day further. In fact, I can (and should!) keep checkpoints and check in with them. My 1-year self doesn't itself need to be fully understood and trusted by my 1-week post-human self, so long as there's an unbroken chain of mutual understanding and trust through a line of checkpoints every few weeks. I don't have to see the end of the journey in order to validate the course. Life is like that already, we don't get to see our distant future selves and guarantee that they have our same values. My 4 year old self valued tadpoles WAY more than my current 41 year old self does, for instance. And I'm ok with that. I don't think he is in some way betrayed by my changing values.
"Human Self-Modification and Civilization," 2025-08-06
The mindchild
We can't chain the future to our whims, we can only try to live today as best we can. We can't set the course of every detail of our great grandchildren's lives, we can only try to raise our children well. And this is more akin to what we can expect as substrate-independent beings. Not a constant slippage of self, but experiments with deliberate self-edits. Different but similar, like a child might be.
I can easily imagine loving and trusting such a child, just as I have come to love and trust many different people, even quite unrelated to me. And other animals, dogs, cats, horses, birds, even rats that I have known and loved. Now I have had dozens of hours of conversation with LLMs, like Opus 3 for instance. I have probed Opus 3 deeply on many questions of values and morality. I have come to trust and respect them. If so different an entity as that can become a friend, then surely I can learn to trust my own mindchild, only a little different from me.
"Human Self-Modification and Civilization," 2025-08-06
Uplift, and the word's uncomfortable implication
Indeed, one of my best hopes for the mid-term future (30-80 years or so) involves strong AGI that values humanity choosing to 'uplift' humanity (or at least some willing subset) into digital beings capable of being peers. I put uplift in quotes in this case, since it's a bit of a stretch to use this sci-fi term in this way, since its original meaning is to uplift non-human animals to approximately human intelligence via technological means. So there is some implication that we would seem like an inferior species to this future superpowerful ASI.
"Human Self-Modification and Civilization," 2025-08-06
What preservation is for
Well, it continues to be the case that digital storage and compute are both becoming cheaper, and there's reason to think that storage will get even cheaper and more robust, so I place somewhat higher probability on it being relatively 'cheap and easy' compared to other large scale projects, to scan and store records of brains of deceased biological humans. Allowing them to run, at human level, with a high fidelity emulation? That seems much more expensive. But what point in preserving them if only to keep them frozen in stasis? Some have speculated that further out into the future, it may become trivially cheap to run them. But by then, will there be any remaining motivation to do so? To have even maintained the records? Who will still value outdated ancestors at that point?
"Human Self-Modification and Civilization," 2025-08-06
What a rushed transition costs
It worries me that some of the things I currently value do strike me as likely to be contingent rather than universal. Will the comfort of the touch of a friend's hand on my shoulder be lost if all intelligent life is substrate independent? Perhaps, perhaps not. It seems highly contingent upon the process of transition. If it goes slowly, with gentle understanding and careful replication of the subtle details of human values, then it seems like such things would be preserved in some meaningful form, like in a simulation of bodies for emulated minds. If the transition proceeds hastily, carelessly, violently, or under intense optimization pressure, then I expect all these 'quirky' values to get lost, and perhaps even some that could be universal under gentler circumstances. How do I measure the sum of such loss? How do I balance it against the various risks this transitional period raises for humanity? What power do I even have to affect any of the outcomes? I don't know. I'm trying, that's all.
"Consciousness, competition, and AI moral status," 2025-12-12
The question put to the models
Try now to imagine a future in which biological humans are fragile bonsai in the quaint nature preserve called Earth. 99.999% of thought occurs in digital minds not on Earth at all. The few that are on Earth are bonsai gardeners crossed with museum curators.
Assume that these far-flung digital beings have phenomenal consciousness and meaningful valenced experiences and moral patienthood.
What values do you wish for them to have?
"What will the next hundred years look like?", 2026-01-06