@philfung

**pfung** @philfung 2026-02-26 I read this paper and its awesome - it creates a high-performing, smooth reward function (far superior to GVL) that is SUPER simple to implement with an LLM. IMPLEMENTATION: 1\. SELECT A MODEL: Pick an open-weight, multimedia LLM (ie Qwen3-VL). 2\. PROMPT THE MODEL: Send the LLM the following prompt: "The above video shows a robot manipulation trajectory that completes the following task: {INSTRUCTION}. Decide whether the above statement is True or not. The answer is: " \[where INSTRUCTION is any task like "fold the towel" or "pour coffee into the cup"\] 3\. EXTRACT THE REWARD: Find the \*logit probability\* for the specific token "True" and use that as your reward signal. \[The logit probability is the raw, unnormalized score assigned by the model to the "True" token before it passes through the softmax layer. This logit prob is available for open-source models and some closed-source models - for example, ChatGPT exposes log probs, whereas Claude does not\] That's it!! Obviously the logit prob and using the term "True" are key insights. It is quite elegant. Congrats to the brilliant authors at @UW and @allen\_ai ! > 2026-02-26 > > Instead of asking a VLM to output progress, it reads the model’s internal belief directly from token logits. No in-context learning. No fine-tuning. No reward training. 📈 > > We introduce: TOPReward, a zero-shot reward modeling approach for robotics using token probabilities from --- **Ville** @VilleKuosmanen [2026-03-02](https://x.com/VilleKuosmanen/status/2028391026851737658) Have you tried replicating it? Sounded like something that wouldn’t be too hard to replicate without source code and the VLM seemingly works without post-training (except may need normalising the preds) so could be great for zero-shot generalisation🤔 --- **pfung** @philfung [2026-03-02](https://x.com/philfung/status/2028494365820436525) Yes you're right def worth doing --- **Sadi Moodi** @MoodiSadi [2026-03-02](https://x.com/MoodiSadi/status/2028340143506280942) This is brilliant! Using token logits directly as reward signals is such an elegant solution. Key insight: the logit prob for "True" before softmax gives you a smooth, differentiable reward. Two things to consider: 1) Make sure your instruction templates are diverse enough to --- **Andrew Carr** @andrew\_n\_carr [2026-03-02](https://x.com/andrew_n_carr/status/2028348858863087634) Yeah, super clever --- **Jiafei Duan** @DJiafei [2026-03-02](https://x.com/DJiafei/status/2028317075933180086) Thanks! Often times the simplest idea works the best! --- **yunus gsnefreti** @yunus\_eee [2026-03-02](https://x.com/yunus_eee/status/2028496279827845623) using logit probs for retrieval reranking was a thing for llms. I haven’t check the paper but this usually requires a normalizatin/calibration procedure since model has its own biases and outputs can be independent from the input, time to time