← All topics

accounting

2 captures, most recent first.

@aden_barton

— saved image

aden barton (verified, M badge) @aden_barton · 13h
New research: we hired 12 CPAs to complete four realistic accounting tasks, then had AI models from the past few years complete the same work.

As of May 2025, accountants outperformed AI. Now they lose out to today's frontier models, which are near perfect on these tasks. [thread emoji]

[attached chart]
Title: AI models passed accountants on these tasks in 2025 and now score at or near 100%
Subtitle: Accuracy of AI models on the study's accounting tasks by model release date, compared with accountants without AI
Y axis: Accuracy (share of rubric criteria met), 0% to 100%. X axis: Model release date, Jul 2024 to Jul 2026.
Horizontal line: "Average accountant score (without AI assistance): 37%" with a shaded band around it.
Labelled points: GPT-4o, Gemini-2.5 Pro, OpenAI o3, GPT-5, Qwen3.5-122B, Opus 4.8, Opus 5. Dashed linear-fit line rising from about 0% at Jul 2024 to about 100% at Jul 2026.
Footnote: Every model tested is shown, including budget and open-weight variants, which is why several recent releases sit below the accountant line. Scores are averaged across the 4 study tasks. Error bars and the shaded band are 95% bootstrap intervals across runs. Dashed line is a linear fit across all models, drawn between 0% and 100%.
Note from Claude Sonnet 5

Tweet announcing a study comparing 12 CPAs against AI models on four accounting tasks, with a scatter chart of model accuracy by release date against the 37% average accountant score; frontier models near 100% by mid-2026.

ai capabilitiesaccountingbenchmarkslaborchart

@mercor

— saved image

Ted Sanders reposted
Mercor @mercor · 10h
GPT-6 Astra passes more tasks on APEX-Accounting than any other model.

13.1% Pass@1 (#1)
60.0% mean score (#2)

Pass@1 is the proportion of tasks that a model scores 100% at least once across four attempts. Astra passes 56% more tasks than GPT-5.6 Sol and 12% more tasks than Fable 5.1.

Most academic benchmarks measure model capabilities that are misaligned with real work. APEX benchmarks measure what enterprises actually care about.

We built APEX-Accounting with @RampLabs to see if agents can handle a real company's books. Agents work the ledger in QuickBooks, tie it to bank statements in PDFs, chase figures across spreadsheets, and judge what is a real discrepancy. Results are graded against 2,186 criteria created by real accountants.

See full leaderboard: mercor.com/apex/apex-acco...

[embedded chart: "OpenAI GPT-6 Astra - APEX-Accounting leaderboard ranked by Pass@1", Pass@1 with confidence intervals:
GPT-6 Astra (Max) 13.1% ± 4.7%
Fable 5.1 (Max) 11.7% ± 4.3%
Opus 5 (Max) 10.3% ± 4.0%
Fable 5 (Max) 9.5% ± 3.7%
Gemini 3.8 Flash (High) 8.7% ± 3.7%
GPT-5.6 Sol (Max) 8.4% ± 3.5%
Callout: +56.0% more tasks passed. Footer: Pass@1, Harness: Loop, mercor.com/apex]
Note from Claude Sonnet 5

Repost of Mercor's announcement that GPT-6 Astra tops its APEX-Accounting benchmark (built with Ramp Labs) on Pass@1, with an embedded bar chart of six models and an on-screen floating compose button overlapping its lower right.

apex-accountingmercorgpt-6 astrabenchmarkenterprise agentsaccounting