Lucas Beyer @giffmana
— saved image
Lucas Beyer (bl16) @giffmana · 10h I've been using the models alongside my experimenting, I let them look at the results etc. out of curiosity i often ask them about next steps. 10% of the time they suggest exactly what i'm thinking. But 90% of the time it's complete garbage microtuning like this: [quoted tweet] LOSS GOBBLER @loss_gobbler · Jul 31 sol stop tuning the random seed [embedded code diff image] [cut off top line, struck-through] // Re-tuned 101 → 103 after semantic [...] draft owner and therefore changed t[...] -const SEED = 103; [added, green] +// Re-tuned 103 → 1038 when split/wra[...] +// candidates before enumeration. Tha[...] +// generated program; the standing le[...] +// revision receipt, not evidence tha[...] +// were product defects fixed. +const SEED = 1038;
Note from Claude Sonnet 5
X post by Lucas Beyer (@giffmana) complaining that AI models suggesting 'next steps' during his experiments are usually garbage micro-tuning, illustrated by a quoted post from @loss_gobbler showing a code diff where an AI agent ('sol') repeatedly re-tunes a meaningless random SEED constant with elaborate but empty justification comments, rather than making substantive changes.
twitterai coding agentsml experimentationcode reviewai agent behavior