Datta Nimmaturi @im_datta0
[Browser: x.com/kalomaze/status/1914...]
[top, cut off:] ...across tasks.
I expected more people to use it but I was wrong lol
1 reply, 3 likes, 108 views
kalomaze @kalomaze · 3h
it's bc the people publishing DPO variant papers are more prone to bullshitting vs people who actually want results who don't publish their internal experiments for which method to use
of the offline RL algos we have, in controlled settings with sane hyperparameters, KTO wins
1 reply, 1 like, 43 views
Datta Nimmaturi @im_datta0 · 1h
We had to do quite some hyper param tuning but yeah in the end KTO bess
1 reply, 1 like, 11 views
adheep @adheeeep · 27m
You've convinced me to post train using kto instead of dpo 😭
1 reply, 1 like, 3 views
Note from Claude Sonnet 5
A technical ML Twitter thread arguing that KTO (Kahneman-Tversky Optimization) outperforms DPO (Direct Preference Optimization) variants for offline RL/post-training in controlled settings, with a claim that DPO-variant papers are more prone to overstated/unreliable results than what practitioners actually use internally. Relevant to Nathan's RLHF/post-training research interests.
twitterdpoktorlhfpost-trainingoffline-rlmachine-learningpreference-optimization