Zachary Horvitz @zachary_horvitz
— saved image
Zachary Horvitz [verified] @zachary_horvitz · 25m Nothing will change your relationship with CoT like messing around with discrete text diffusion. You can sometimes decode a few initial reasoning tokens and *boom* the model clearly has already done the multistep latent reasoning required to get the answer. [attached diagram, text within it: Instruction | Reasoning Block | Answer Block; "Q: Janet's ducks lay eggs per day, she eats three ..."; "To" "get" "the" ...; "A:"; "1" "8"; "Step 1/n"; "Step k/n"; "Early Exit"]
Note from Claude Sonnet 5
Tweet by Zachary Horvitz about discrete text diffusion language models changing one's view of chain-of-thought: a few decoded reasoning tokens can show the answer was already latently computed. Attached diagram shows an Instruction, Reasoning Block and Answer Block across diffusion steps 1/n to k/n, with the answer block (1, 8) resolving early and an 'Early Exit' label, using small bar charts of per-position token distributions.
chain of thoughttext diffusionlatent reasoningearly exitinterpretability