secemp @secemp9 · 3h
one thing I noticed recently, while it's true for some tasks, depending on complexity, SFT alone is enough (+ RL ofc)
but for really small models, like say 1B, I noticed I could get pretty close to what I wanted if I used SFT+DPO+KTO on the same model
if I used KTO alone, it worked nicely but somehow ended up self explaining everything, DPO alone works but for creative/technical writing, still has some slop depending on the base model, SFT needs a lot more examples
using them in that order almost act as a regularizer without overfitting
Note from Claude Sonnet 5
A practitioner's tweet on training small (~1B parameter) language models, comparing SFT, DPO, and KTO fine-tuning methods and noting that chaining them in sequence acts as a regularizer against overfitting. Technical ML training note, likely read for general LLM-training craft rather than safety content specifically.
