← All topics

best@k

1 capture, most recent first.

@tobyordoxford

— saved image

Santiago M. @sanmking · 19h
Isn't this the complete opposite to that:

[quoted post]
Dimitris Papailio... @Dimitris... · Sep 20
Test-time communication looks like a next axis for scaling capabilities

New paper with the incredible @jon_ghoh and @vkontonis @ShivamGarg91462 and Akshay : ...

[embedded figure: three line charts: "ARC-AGI-3 (public)" (x: Agents (k), y: Tasks solved; series Best@k Sonnet 4.6 vs Team@k Sonnet 4.6; labels 6.6x, 8.0%, 4.3x, 4.6%), "FrontierCS Polyomino Packing" (x: Time (hours), y: Packing score; series Best@4 Opus 4.6 vs Team@4 Opus 4.6; labels prior best 0.894, 0.922, 0.893), "MNIST Compression" (x: Output tokens, y: Compression size (bytes); series Best@4 GPT-5.6 Sol vs Team@4 GPT-5.6 Sol; labels prior best 2,461 B, 3,160 B, 1,957 B)]
Figure 1: Communication outperforms independent agents in open-ended tasks. With communication, a team of k communicating agents not only outperforms best@k but scales success rate on ARC-AGI-3 faster. On algorithmic and ML optimization tasks, team@k beats single-agent runs and best-known human solutions, yielding new state-of-the-art results.

3 replies, 7 likes, 421 views

Sharmake Farah reposted
Toby Ord @tobyordoxford

Those graphs are showing the difference between swarm scaling and best@k scaling (i.e. what coordination adds) whereas I looked at the difference between swarm scaling and longer CoT (the most common current inference scaling technique).

12:46 AM · Sep 22, 2026 · 283 Views

2 replies, 10 likes, 1 bookmark

Relevant

Toby Ord @tobyordoxford · 9h
I'm not surprised swarm scaling beats best@k scaling on verifiable tasks (where it doesn't there is no value in working together). [cut off]
Note from Claude Sonnet 5

X thread screenshot: Santiago M. asks whether a quoted Papailiopoulos post and figure (test-time communication among agent teams, team@k vs best@k) contradicts Toby Ord's earlier analysis; Toby Ord replies that the graphs measure swarm vs best@k scaling while he compared swarm scaling to longer chain-of-thought. A reply of his is cut off at the bottom by the app UI.

inference scalingswarm scalingmulti-agentbest@kchain of thoughttoby ordarc-agi-3