best-of-n — Timeline

2026-09-22 — saved image

Jongho Park @jon_ghoh

We did run those in the arc-agi-3 ablation here! See the solid green line. Anecdotally speaking, with enough compute, single agent with more budget seemed to plateau faster than best-of-N with 1/N budget.

[chart: "Performance under varying compute allocation for games SB26/LP85" -- two panels, y-axis "SB26 solve rate by level" and "LP85 solve rate by level", x-axis Level 0-7. Legend: Team@5 (1x budget per agent); Team@5 (0.2x budget per agent); Best@5 (1x budget per agent); Best@1 (5x budget per agent); Best@1 (1x budget per agent)]

10:28 AM · Sep 21, 2026 · 5,409 Views
1 reply, 1 repost, 12 likes, 6 bookmarks

Dimitris Papailiop... @DimitrisP... · 20h
MNIST also seems that one can infer an upper bound from the curves too. Takes 10-100x for the single best agent to get to the team's level.

[chart: "MNIST Compression vs. Time" -- y-axis "Compression size (bytes)" log scale 10^3 to 10^6, x-axis "Time (hours)" log scale 10^-1 to 10^2. Legend: Single-agent runs; Team@4 GPT-5.6 Sol; Best human compression (dotted line, labelled 2,461 B). Team line ends at 1,957 B.]

Note from Claude Sonnet 5

Quote-tweet style screenshot of two X posts about team-of-agents versus single-agent compute scaling, each with an attached chart (solve rate by level on ARC-AGI-3 games SB26/LP85; MNIST compression size vs time with a team line dropping below the best human compression).

agents compute scaling best-of-n arc-agi-3 multi-agent teams mnist compression

@abeirami

2026-08-03 — saved image

Ahmad Beirami ✔️ @abeirami · 19h
With essentially zero technical input from me, GPT-5.6 Sol and Fable 5 not only proved a conjecture we left open ~2 years ago on best-of-n, but also delivered a strictly tighter bound with a clean and insightful derivation.

We are officially in a new era of mathematical reasoning!

Much of what previously counted as meaningful technical contribution is now routine for these models. This level of reasoning is being fundamentally democratized.

The kind of research that used to take months to become a paper is now achievable in minutes.

[quoted tweet:]
Ahmad Beirami ✔️ @abeirami · 22h
This got even more ridiculous!

I was trying to use the context of this session to nudge Sol to improve another result. Instead, it misunderstood me as wanting to improve this …

[attached images: two page-scan panels of a math writeup titled 'A sharper finite-atom KL bound for best-of-n', with theorem statement, proof sketch, and a plot comparing an analytical formula, an estimator, a sharpened estimator, and exact KL divergence across a range of n]

Note from Claude Sonnet 5

Tweet from Ahmad Beirami reporting that AI models 'GPT-5.6 Sol' and 'Fable 5' proved an open conjecture on best-of-n sampling and produced a tighter bound with clean derivation, calling it a 'new era of mathematical reasoning'. Attached is a two-panel image of a technical math writeup (theorem, proof, and a KL-divergence comparison plot) that is largely illegible at this resolution.

ai math gpt fable best-of-n twitter mathematics model names