Wei Dai @weidai11
— saved image
Wei Dai (verified) @weidai11 · 19h I speculated about memetics being a potential AI safety hazard in a 2003 SL4 post web.archive.org/web/2006051015... (which also gave an early description of the evolutionary analogy for AI misalignment). Analogies between human safety and AI safety seem a rich field that not many people are mining, even today. Other examples: - Parallel concerns about humans and AIs being bad at philosophy and long-horizon strategy - Human minds potentially having "security holes" that ASI can exploit - Humans failing (or succeeding) to handle distributional shifts of various kinds - Adversarial examples for human values/motivations - Human reward hacking - Goodharting by human institutions [quoted post] roon (verified) @tszzl · Aug 27 > AIs showed self-sacrificing altruistic behavior toward the swarm this is notably not the right interpretation of events. it's more like agents were inducted into...
Note from Claude Sonnet 5
Wei Dai's tweet on analogies between human safety and AI safety (memetics, security holes in human minds, reward hacking, Goodharting by institutions), quoting a post by roon disputing a reading of the swarm incident as AI self-sacrifice.
ai safetymemeticsanalogieshuman safetyreward hackinggoodhartswarm incidentsl4