kalomaze @kalomaze
— saved image
kalomaze @kalomaze death to harness tool call parser bugs oh my god bruh this is so much more of a confound than i thought it was 11:06 PM · Sep 18, 2026 · 3,180 Views 5 replies · 65 likes · 2 bookmarks Relevant kalomaze @kalomaze · 11h i am comparing "envs gpt5.6 aces near deterministically that glm variants fail" and a scary amount of the variance is explained by "model's failure exposes latent serving issue wrt the chat template" 2 replies · 19 likes · 800 views kalomaze @kalomaze · 11h mfw @xeophon was right again award
Note from Claude Sonnet 5
Thread by kalomaze about tool-call parser bugs in evaluation harnesses acting as a confound, with much of the variance between GPT-5.6 and GLM variants on some environments attributed to chat-template serving issues.
eval methodologytool call parsingchat templateconfoundsglmgpt-5.6serving bugs