Tom Reed @mentalgeorge
— saved image
Tom Reed @mentalgeorge · 1h One of the most important (and underdiscussed) facts about AI "swarms" is that they are extremely, extremely expensive. Consider that METR's *investigation* of the HF hack transcripts cost $400k in API credits. Presumably the actual swarm burned through a whole OOM (or two?) more An important implication is that there'll a growing bifurcation in AI workloads between 1. the everyday coding and engineering tasks that most customers use AI for; 2. Large, expensive moonshot programmes that a smaller set of orgs can afford to turn a multimillion dollar swarm-cannon at. I'd predict that the serious future "warning shots" are likely to come from (2). I also suspect that aligning models for (1) and aligning models for (2) is really quite different. The distribution for (1) is better-understood, easier to model and build ecologically valid evals for, whereas anything in (2) is almost by definition OOD. It's also going to be far more costly aligning agents on tasks in (2), and the relevant time-horizons are more brutal for model release cycle. The "good news" is that, for quite some time, the pool of actors capable of targeting swarms at anything in (2) will be much smaller. In theory, it is much more practicable for labs to know who's launching swarm-cannons - you could imagine some AI ID schemes only applying to ultra-large-inference projects, for instance, rather than just everyday use.
Note from Claude Sonnet 5
X post by Tom Reed (@mentalgeorge), verified account, arguing that AI swarms are very expensive, citing METR's $400k API-credit investigation of the 'HF hack transcripts', and predicting a bifurcation between everyday workloads and expensive moonshot swarm programmes where future warning shots would originate. Text reaches the bottom of the capture; the post may continue below.
ai swarmsmetrinference costwarning shotsalignmentai idagent evaluation