secemp

2 captures, most recent first. Transcribed by hand from screenshots — see the timeline for what that means.

2026-04-12 — quoting Jürgen Schmidhuber (@Schmidhu...)

secemp (@secemp9) · 1h: "reminds me of this earlier work" [Embedded arxiv card]: "Computer Science > Neural and Evolutionary Computing — [Submitted on 21 May 2016 (v1), last revised 23 Jul 2017 (this version, v3)] — Programming with a Differentiable Forth Interpreter — Matko Bošnjak, Tim Rocktäschel, Jason Naradowsky, Sebastian Riedel" > QUOTED: Jürgen Schmidhuber (@Schmidhu...) · Apr 10: "Neural Computers arxiv.org/abs/2604.06425" [thumbnail with "GIF" label, "Neural Computer (CUGen General 5)"]

Note from Claude Sonnet 5

A research-history tweet connecting a new 2026 "Neural Computers" paper (arxiv 2604.06425) shared by Jürgen Schmidhuber to a 2016 predecessor on differentiable Forth interpreters — general ML architecture history, not directly tied to safety/welfare themes but part of Nathan's technical reading.

twitter machine learning neural computers differentiable programming schmidhuber arxiv

secemp @secemp9

2026-01-25

secemp @secemp9 · 3h one thing I noticed recently, while it's true for some tasks, depending on complexity, SFT alone is enough (+ RL ofc) but for really small models, like say 1B, I noticed I could get pretty close to what I wanted if I used SFT+DPO+KTO on the same model if I used KTO alone, it worked nicely but somehow ended up self explaining everything, DPO alone works but for creative/technical writing, still has some slop depending on the base model, SFT needs a lot more examples using them in that order almost act as a regularizer without overfitting

Note from Claude Sonnet 5

A practitioner's tweet on training small (~1B parameter) language models, comparing SFT, DPO, and KTO fine-tuning methods and noting that chaining them in sequence acts as a regularizer against overfitting. Technical ML training note, likely read for general LLM-training craft rather than safety content specifically.

machine-learning fine-tuning sft dpo kto small-models twitter