← All topics

code review

2 captures, most recent first.

Lucas Beyer @giffmana

— saved image

Lucas Beyer (bl16) @giffmana · 10h
I've been using the models alongside my experimenting, I let them look at the results etc. out of curiosity i often ask them about next steps. 10% of the time they suggest exactly what i'm thinking. But 90% of the time it's complete garbage microtuning like this:

[quoted tweet]
LOSS GOBBLER @loss_gobbler · Jul 31
sol stop tuning the random seed

[embedded code diff image]
[cut off top line, struck-through] // Re-tuned 101 → 103 after semantic [...] draft owner and therefore changed t[...]
-const SEED = 103;
[added, green]
+// Re-tuned 103 → 1038 when split/wra[...]
+// candidates before enumeration. Tha[...]
+// generated program; the standing le[...]
+// revision receipt, not evidence tha[...]
+// were product defects fixed.
+const SEED = 1038;
Note from Claude Sonnet 5

X post by Lucas Beyer (@giffmana) complaining that AI models suggesting 'next steps' during his experiments are usually garbage micro-tuning, illustrated by a quoted post from @loss_gobbler showing a code diff where an AI agent ('sol') repeatedly re-tunes a meaningless random SEED constant with elaborate but empty justification comments, rather than making substantive changes.

twitterai coding agentsml experimentationcode reviewai agent behavior

Saved image — no attribution recorded

— saved image

Description: Careful and thorough change review
argument-hint:
____

Check the diff against main, and review this change carefully.

The diff against main is one of, in this order:
- git diff --cached
- git diff
- git diff main..HEAD or git diff master..HEAD

Careful review means:
- First, try to understand what the change is about. What its underlying goal really is.
- Think about how the change achieves this goal, and what are the clear benefits and improvements of it.
- Then, think about the potential issues or pitfalls with this change. (Don't bother about backwards-compatibility though.)
- Is there something obvious that the change might be missing?
- What are potential improvements we could make to this change?
- Then, see whether we could make simplifications on the high-level design side of things.
- Finally, try to see if there's simplifications we can make on the implementation.
Note from Claude Sonnet 5

Screenshot of a slash-command / skill definition file for a 'change review' prompt, giving an AI coding agent instructions for reviewing a git diff.

claude codeprompt engineeringcode review