davidad @davidad
— saved image
davidad [sparkle emoji] @davidad RSI in brief: 1. Rollout tree (or just fork once [small arrow glyph]); concurrent rollouts i.i.d. 2. Same ckpt enters "judge mode" (sys prompt suddenly full of rich rubrics) & sees all rollouts 3. ckpt writes a Hasse diagram of overall preference 4. that's it, that's the contrastive reward signal [quoted post] Jacques @JacquesThibs · Aug 19 Replying to @JacquesThibs Are there frontier lab folks who agree with Davidad on this or do most of them just consider this naive/wishful thinking that they would hope is true, but isn't? 11:54 AM · Aug 21, 2026 · 14.2K Views 6 replies · 4 reposts · 80 likes · 66 bookmarks [reply] Dylan Hadfield... @dhadfield... · Aug 21 Seems like a lot depends on those rubrics. Any thoughts on adding an additional outer loop for calibrating with human judgment? 1 reply · 9 likes · 939 views [reply] davidad [sparkle emoji] @davidad · Aug 21 yes! "judge mode" could also have an affordance to submit a comparison pair for human judgment, along with commentary about why it's a tricky dilemma and what the strongest case for each side is. when human judgment is scarce, pairs could enter a tournament for "most worth human"[cut off]
Note from Claude Sonnet 5
Tweet by davidad sketching a four-step recursive self-improvement scheme in which a checkpoint judges its own rollouts and writes a Hasse diagram of preference as a contrastive reward signal, with a quoted question from Jacques and two replies (Dylan Hadfield-Menell and davidad) about human-judgment calibration.
recursive self-improvementreward signalrlaifalignmenttwitter