80000-hours

Rob Wiblin @robertwiblin Even 'aligned AGI' naturally kills democracy and leads to oligarchy, or worse. That's the take of Anthropic's past alignment evals team lead, Prof @DavidDuvenaud. Once humans aren't needed to do jobs or serve in the military, to governments we look like "meddlesome parasites". With voters unable to contribute but engaged in incessant activism to extract resources from others – resources the country needs to avoid domination by rivals – the attraction of mass disenfranchisement could be overwhelming. In 2025 David co-authored "Gradual Disempowerment", which aimed to lay out this and many other political, economic, and cultural forces that could sideline ordinary people (and maybe all people) in the presence of machines that can cheaply do everything humans will do. Most controversially, David and colleagues believe that competitive forces will compel disempowerment, even if all those AIs are aligned and loyal to their users. I wasn't sure how much I believed this vision of how the future might play out, so I interviewed him for The 80,000 Hours Podcast to probe how well it holds up. He and I covered: [cut off]