← All topics

multi-agent evals

1 capture, most recent first.

@leo_linsky

— saved image

Leo Linsky @leo_linsky · 11h
We evaluated GPT-6-Astra in 100 complex, unsaturated multi-agent coding environments, competing and cooperating with other models in open-ended tasks.

It's the new frontier model by a landslide. It's even more dominant than the Fable 5 release, because not only does it wipe the floor with the second best model (Fable 5.1), it was also 80% cheaper and 30% faster in agentic coding.

This is a groundbreaking model. The biggest breakthrough since Opus 4.5, maybe even since GPT 4.

This is also a testament to how unsaturated our evals are. Some of the open-ended environments we run were introduced almost a year ago and can still compete frontier model submissions against older, deprecated models like Sonnet 4 without signs of saturation. Our newer ones are much more challenging and completely auto-generated.

We are clearly living in a post-AGI world. Do something interesting and useful with it.

[embedded images, left one a leaderboard table titled "GBENCH Intelligence Benchmark — Using complex games to probe real intelligence."; right one a scatter plot titled "...nce Efficiency" (cut off), axis ticks $20 $10 $5.0. Table columns: Model, Provider, GScore, Avg Percentile, Context. Rows visible, model names cut off at left edge:]
T-6 Astra | OpenAI | 81.5 | 89.7% | 1.05M
ude Fable 5.1 | Anthropic | 74.8 | 80.9% | 1.00M
ude Opus 5 | Anthropic | 74.0 | 80.9% | 1.00M
ude Fable 5 | Anthropic | 70.5 | 79.8% | 1.00M
mini 3.8 Flash | Google | 69.7 | 75.0% | 1.05M
T-5.6 Sol | OpenAI | 68.8 | 74.4% | 1.05M
k 4.6 | xAI | 66.6 | 74.0% | 500K
mini 3.7 Flash | Google | 64.9 | 68.7% | 1.05M
mini 3.6 Flash | Google | 64.0 | 70.1% | 1.05M
i K3 | Moonshot | 63.2 | 71.9% | 1.05M
T-5.5 | OpenAI | 62.8 | 71.1% | 1.05M
T-5.6 Terra | OpenAI | 61.7 | 66.7% | 1.05M
Note from Claude Sonnet 5

Dark-mode X post by Leo Linsky reporting GPT-6-Astra results on multi-agent coding environments and calling it a post-AGI moment, with an attached GBENCH leaderboard table (Astra at the top, 81.5 GScore, ahead of Fable 5.1 and Opus 5) and a partly cropped cost-efficiency scatter plot. Table figures are small and the last digit of each score may be a confidence suffix; the chart is kept as an image.

gpt-6astragbenchbenchmarksagentic codingmulti-agent evalsfable 5.1unsaturated evalspost-agi