@AlexGDimakis

[Repost indicator] Mert Cemri reposted Alex Dimakis ✓ (@AlexGDimakis) — 5h I am very excited about this research: We show 2 things: 1. If you just do random sampling (i.e. you try to solve a problem k times independently, and keep the best) your ELO scaling will be linear in log(test-time-compute). Agents like Claude-Code and Codex scale like that after a few hours. 2. We compare human expert coders to coding agents on the same tasks (from AtCoder Heuristic Contest). The exciting finding is that humans scale super-linearly. This is evidence that humans do continual learning, while they are solving a problem! I.e. they learn more about the coding problem they are trying to solve and scale fundamentally better compared to randomly trying things in a memoryless fashion. This is empirical evidence that supports what many of us have felt for a while: unless we solve continual learning we will not be able to outperform humans in tasks that take many days. Current coding agents are not able to do this. > QUOTED: Qiuyang Mang ✓ (@MangQiuyang) — 9h > (1/n) New blog from UC Berkeley, UW, and Princeton: Who scales better in long horizon: AI coding agents or top coders? > We compared modern agents to top human ... > [Embedded chart: "Human vs Agent" — Y axis "Elo Rating" (1000–1800), X axis "time" (1h to 7d). Lines: "top10-humans" (dark blue, climbs steadily to 1853 at end), "top50-humans" (light blue, climbs to 1587), "Claude Code Opus-4.6" (orange dashed, sprints early to ~1348 by 24h then plateaus), "Codex GPT-5.5" (green dashed, flat near 1092). Annotations: "agents sprint early", "then plateau by 24h", "humans keep climbing for days".]