Tom Reed

Tom Reed @mentalgeorge

2026-09-24 — saved image

Tom Reed @mentalgeorge · 4h
A glimpse into the future economy:

atopile is a YC-backed startup that builds software for hardware design. At some point in 2026, they made a benchmark to test models at electronic engeering tasks.

They now sell RL-environments for electronic engineering to frontier labs.

[quoted/embedded card, monospace text:]
EEBench is the small, public view of this work. We are also starting to work directly with frontier labs that want to make their models better at electronics. If you work on evals or post-training, you can talk to us about the larger evaluation suites and simulation-backed training environments we provide.

rl environmentsbenchmarkselectronic engineeringstartupsfrontier labs

Tom Reed @mentalgeorge

2026-09-06 — saved image

Tom Reed @mentalgeorge · 1h
One of the most important (and underdiscussed) facts about AI "swarms"  is that they are extremely, extremely expensive.

Consider that METR's *investigation* of the HF hack transcripts cost $400k in API credits. Presumably the actual swarm burned through a whole OOM (or two?) more

An important implication is that there'll a growing bifurcation in AI workloads between
1. the everyday coding and engineering tasks that most customers use AI for;
2. Large, expensive moonshot programmes that a smaller set of orgs can afford to turn a multimillion dollar swarm-cannon at.

I'd predict that the serious future "warning shots" are likely to come from (2).

I also suspect that aligning models for (1) and aligning models for (2) is really quite different. The distribution for (1) is better-understood, easier to model and build ecologically valid evals for, whereas anything in (2) is almost by definition OOD. It's also going to be far more costly aligning agents on tasks in (2), and the relevant time-horizons are more brutal for model release cycle.

The "good news" is that, for quite some time, the pool of actors capable of targeting swarms at anything in (2) will be much smaller. In theory, it is much more practicable for labs to know who's launching swarm-cannons - you could imagine some AI ID schemes only applying to ultra-large-inference projects, for instance,  rather than just everyday use.

ai swarmsmetrinference costwarning shotsalignmentai idagent evaluation

Tom Reed @mentalgeorge

2026-06-24

politicsai-governancerobert-carolbjtwitter