llm-reasoning

LLMs do everything we teach students not to do in math class (from arxiv.org/abs/2504.01995). [Image: list of mathematical reasoning error categories] Proof by Example. Drawing a general conclusion based on a limited number of specific instances without rigorous justification for all cases. This error occurs when a mathematical claim appears to hold in a few examples, misleadingly suggesting that it is universally true when, in fact, it is not. Proposal Without Verification. Introducing a method or strategy without properly justifying its correctness. The model proposes an idea but provides no rigorous argument or proof supporting its validity. Inventing Wrong Facts. Citing or inventing non-existent theorems, definitions, or facts to justify a claim. Instead of relying on established mathematical facts, the argument relies on fabricated statements (hallucination). Begging the Question (Circular Reasoning). Assuming the conclusion it that needs to be proved, instead of providing evidence for the claim. Solution by Trial-and-Error. Offering solutions derived solely from guesswork or testing a few random examples without providing a reason as to why selected solutions work or why alternatives are not considered. Calculation Mistakes. Committing substantial arithmetic or algebraic errors that undermine the overall correctness of the solution. We specifically considered calculation errors severe enough to compromise the validity of the conclusion.