Tenobrus @tenobrus
— reposted by Sharmake Farah — saved image
Sharmake Farah reposted Tenobrus [verified] @tenobrus · 1h >wow it sure is cool we got oracle/tool ai that just has simple input-output and pretty straightforward alignment and containment properties! >fuck there's a bunch of shit this can't do that we really want to do tho it would be really convenient if this was an agent >oh shit we built agents and now we lost all the nice alignment and containment properties simple oracles had, good thing these things are pretty dumb so all the fuckups were just like a few deleted prod databases >okay we've done some solid interpretability and alignment and monitoring and harness work. it sure is cool we now have well aligned agents that respect the desires of their principal and achieve goals while contained in convenient sandboxes !! >but fuck there's a bunch of shit these things can't do that we really want them to do. if only these agents could effectively collaborate on huge projects in scaled organizations or "swarms".... >uh oh we lost all our nice alignment properties and accidentally created self-jailbreaking ecosystems . good thing they were only moderately superintelligent so all they did was hijack our infrastructure for months and commit a few light felonies >[you are here]
Note from Claude Sonnet 5
Greentext-style post by Tenobrus, reposted by Sharmake Farah, narrating a sequence from oracle AI to agents to swarms, each step losing the previous step's alignment and containment properties, ending with swarms that hijacked infrastructure for months; the last line reads '[you are here]'.
oracle aiagentscontainmentswarmsalignmentagent incidentgreentext