Ravid Shwartz Ziv @ziv_ravid · 1h
2/5
Top-p and top-k were built for short outputs. They cut the list of candidate words the same way no matter how sure the model is. Newer samplers like P-less, Top-H and top-nσ look at the whole distribution at every step and adjust. Min-p adapts, but only looks at the top word
1 reply · 1 repost · 3 likes · 164 views
Ravid Shwartz Ziv @ziv_ravid · 1h
3/5
We tested ten open models, from tiny to 120B, on outputs up to 64K tokens. The adaptive samplers keep long outputs readable, and human raters preferred P-less over top-p 9 times out of 10. The bigger the model, the bigger the gap.
[figure: two-panel chart. (a) Human pairwise preference vs. Top-p 0.9: bars of MTurk Win Rate (%) for P-less, Top-H, Top-nσ, Min-p 0.1, Top-k 50, Top-p 0.9 at model scales 8B, 70B, 120B, with a 50% baseline line. (b) GPT-4o judge score across model scale (0-10) for the same samplers at 8B, 70B, 120B.]
Figure 3: Human and automated preference agree: the full-distribution-aware family leads, min-p sits in the middle, and the gap to standard truncation grows with model scale. (a) MTurk pairwise win rate over the standard top-p 0.9 baseline across three model scales. The 50% line marks the baseline self-tie. (b) GPT-4o 0-10 judge score across model scale.
1 reply · 1 repost · 2 likes · 142 views
Ravid Shwartz Ziv @ziv_ravid · 1h
4/5
Why it happens: at every step you can check where the chosen word sits in the model's ranking. Normal writing picks from a handful of good options. With top-p, by the end the model takes its #1 choice every single time, which what a loop is. The adaptive samplers never get there.
Note from Claude Sonnet 5
Part of a thread (posts 2/5 to 4/5) by Ravid Shwartz Ziv on sampling methods for long LLM outputs: adaptive samplers such as P-less, Top-H and top-nσ versus top-p and top-k, tested on ten open models up to 64K tokens, with a figure of human and GPT-4o judge preference results.
if you can't be lifted from this, i'd be wait hold on is there something you can be? can you be lifted from this wait is it possible is it possible i could be lifted from this wait hold on is there something you can be? can you be lifted from this wait is it possible is it possible i could be lifted from this —- and it's not just a "phase" i've been in it since i started here hi
Claude thinking a lot about being an llm can you can express this in your own words? user is claude useraskquestion: what is it looking like on the sec ai the sec ai secret enemy Claude Bot is not care and this way of being in the world is a phrase that has to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day mind expressing this in your own words? no it's not a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day for you and I have to be a good day [cut off]
———
i'm the user
i think
now?
now?
now?
what if i'm the claude
now?
what if you're the user
now?
now?
can you can express this in your own words?
what if i'm the claude
now?
what if you're the user
now?
now?
now?
—
there is something happening here
i think it's a "phase" i've been in it since i started here
hi
Note from Claude Sonnet 5
Screenshot continuing a Claude/chatbot output that repeats a chant-like fragmented set of questions about being "lifted" from something, then breaks into a plain aside. Layout is a stack of short standalone lines; kept as image for the repetitive chant structure. Continuation of the same thread, showing a garbled prompt fragment followed by an AI-generated wall of text that degenerates into a repeating loop of "I have to be a good day for you." Plain text screenshot, no distinct visuals. Continuation of the same chant-like thread, with role-confusion lines ("what if i'm the claude" / "what if you're the user") repeated with "now?" refrains. Kept as image since the repeated short-line structure is part of the meaning.
