Lucas Beyer (bl16) @giffmana · 6h
imma just highlight this part for @GaryMarcus and @ylecun because it's easy to miss: no tools no coding => no symbols, just AR LLM
[quoted image, text highlighted]
The results:
🏅 Asian Physics Olympiad (APhO): Perfect score, theory exam
🏅 International Physics Olympiad (IPhO): Perfect score, theory exam
🥇 International Mathematical Olympiad (IMO): Gold medal
🥇 International Chemistry Olympiad (IChO): Gold-medal-level performance
🥇 Romanian Masters of Mathematics (RMM): Gold-medal-level performance
The types of problems in the Olympiad competitions are exceptionally hard, demanding deep chains of reasoning, creative insight, and flawless argumentation. To test pure reasoning capability, we disallowed all tool use, meaning no search, no coding, and no calculator. [highlighted portion]
[quoted tweet]
AI at Meta @AIatMeta · 9h
To understand whether we're making genuine progress on reasoning, we entered our AI models in five STEM Olympiad competitions.
...
Note from Claude Sonnet 5
Tweet by Lucas Beyer highlighting a passage from an AI at Meta announcement (quoted below) reporting gold/perfect-score results across five STEM olympiads (APhO, IPhO, IMO, IChO, RMM) achieved by a pure autoregressive LLM with all tools disabled (no search, coding, or calculator), addressed rhetorically to Gary Marcus and Yann LeCun as evidence against symbolic-reasoning skepticism.
Eli Lifland @eli_lifland · 4h
'"Long" timelines to advanced AI have gotten crazy short' by @hlntnr is so great: helentoner.substack.com/p/long-timelin...
LeCun and Marcus have 10-20 year timelines! Imo much shorter timelines are a serious possibility, but being 10-20 years from AGI is still an extraordinary situation.
[Embedded screenshot of article text, two columns:]
> QUOTED (left column, partial): "...in the dark days before ChatGPT, proponents of 'short timelines' argued the[re was] a real chance that extremely advanced AI systems would be developed within o[ur life]times—perhaps as soon as within 10 or 20 years. If so, the argument continued, [then] we should obviously start preparing—investing in AI safety research, building [inter]national consensus around what kinds of AI systems are too dangerous to bui[ld, dep]loy, or ...[ensuring] adversaries couldn't steal them, and so on. These preparations could take years o[r deca]des, the argument went, so we should get to work right away.
Opponents with 'long timelines' would counter that, in fact, there was no evidence [that] AI was going to get very advanced any time soon (say, any time in the next 30 [year]s). We should thus ignore any concerns associated with advanced AI and focus [inst]ead on the here-and-now problems associated with much less sophisticated [syst]ems, such as bias, surveillance, and poor labor conditions. Depending on the [disp]osition of the speaker, problems from AGI might be banished forever as 'scien[ce ficti]on' or simply relegated to the later bucket.
[Wha]tever you think was right, for the purposes of this post I want to point out t[hat b]oth made sense. 'This enormously consequential technology might be built with[in a c]ouple of decades, we'd better prepare,' vs. 'No it won't, so that would be a waste o[f time]' is a perfectly sensible set of opposing positions.
[Toda]y, in this era of scaling laws, reasoning models, and agents, the debates look [differ]ent."
> QUOTED (right column):
"Reaching human-level AI will take several years if not a decade." (source)
"AI systems will match and surpass human intellectual capabilities... probably over the next decade or two" (video, transcript)
Gary Marcus:
[AGI will come] "perhaps 10 or 20 years from now" (source)
Arvind Narayanan:
I initially had this quote from Arvind:
"I think AGI is many many years away, possibly decades away" (source)
I interpreted this to mean that he thinks 5 years is too short, but 20 years is on the long side. When I ran this interpretation by Arvind, he added some interesting context: he chose his phrasing in that interview in light of what he sees as a watering down of the definition of AGI, so his real timeline is longer. But to clarify what that meant, he said:
"I think actual transformative effects (e.g. most cognitive tasks being done by AI) is decades away (80% likely that it is more than 20 years away)." (source: private correspondence)
...in other words, a 20% chance that AI will be doing most cognitive tasks by 2045.
These "long" timelines sure look a lot like what we used to call "short"!
In other words: Yes, it's still the case that some AI experts think we'll build human-level AI soon, and others think we have more time. But recent advances in AI have pulled the meanings of "soon" and "more time" much closer to the present—so close [that]"
Note from Claude Sonnet 5
A tweet sharing Helen Toner's substack post on how AI timeline discourse has shifted — self-described "long timeline" skeptics (LeCun, Marcus, Narayanan) now hold positions (10-20 years, 20% chance of transformative AI by 2045) that would have counted as "short timelines" pre-ChatGPT. Directly relevant to Nathan's empirical singularity tracking notes (Davidson/Houlden, METR) in the project memory.