← Timeline

Sam Paech

@sam_paech on X

2 captures, most recent first. Transcribed by hand from screenshots — see the timeline for what that means.

Sam Paech @sam_paech

— saved image

Sam Paech [verified] @sam_paech · 13h

New models tested on creative writing:
GPT-6-Astra
Fable 5.1
Muse Spark 1.3
Gemini 3.8 flash

Subjectively, I strongly prefer Muse Spark 1.3 to both GPT-6 and Fable. It just gives you what you asked for, without an obnoxious house style permeating every request.

Fable has converged on its own flavour of claudeslop, and can't write normally. Meanwhile, GPT-6-Astra has forgotten how to write in paragraphs. It's a sad state of affairs!

Tentatively suggest we reinstate the old tradition of reading rollouts with our actual eyeballs.

[attached image: screenshot of the 'Creative Writing v3' leaderboard (EQ-Bench, 'A LLM-judged creative writing benchmark'), columns Model / Abilities / Style / Slop / Repetition / Length / Rubric Score / Elo Score / Sample. Small print; model names and Elo scores as legible, in leaderboard order (cut off at the bottom):
gpt-6-astra 2163.9
claude-fable-5-1 2152.7
claude-opus-5 2120.6
kimi-k3 2070.6
GLM-5.3 2064.1
gpt-5.6-sol 1963.4
ox-alpha 1960.6
claude-fable-5 1934.6
muse-spark-1.1 1916.1
claude-opus-4-7 1907.1
muse-spark-1.3 1905.5
gpt-5.6-terra 1850.3
gpt-5.5 1843.5
Qwen3.8-2.4T-A95B 1840.9
gpt-5.4 1835.6
claude-opus-4-8 [Elo illegible]
muse-spark-1.2 [Elo illegible]
gpt-5.6-luna [Elo illegible, cut off]]
Note from Claude Sonnet 5

Dark-mode tweet screenshot: Sam Paech on creative-writing tests of GPT-6-Astra, Fable 5.1, Muse Spark 1.3 and Gemini 3.8 flash, preferring Muse Spark for lacking a house style and calling for reading rollouts by eye. Embeds a screenshot of the EQ-Bench Creative Writing v3 leaderboard (small table with orange Elo bars), kept as an image because the numeric columns are too small to transcribe reliably.

creative writingbenchmarkseq-benchgpt-6-astrafable 5.1muse sparkslophouse stylellm judge

Sam Paech @sam_paech

Sam Paech @sam_paech · 7h Been seeing some chatter that the new mistral small 3.2 writes a lot like deepseek v3. This analysis of their slop profiles confirms. I think the network representation here makes a bit more sense than the phylo tree, given the complicated nature of model lineages. [Two embedded figures: left is a force-directed network graph of many model nodes; right is a phylogenetic-tree-style dendrogram listing models including claude-sonnet-4, claude-opus-4, anthropic/claude-3.5-haiku-20241022, claude-3-5-sonnet-20241022, claude-3-7-sonnet-20250219, CohereForAI/c4ai-command-a-03-2025, gpt-4o-mini, liquid/lfm-7b, meta-llama/Llama-4-Scout-17B-16E-Instruct, meta-llama/Llama-4-Maverick-17B-128E-Instruct, meta-llama/Llama-3.2-1b/3b-instruct, meta-llama/Llama-3.1-405b/70b/8b-instruct, openai/gpt-3.5-turbo-0613, anthropic/claude-3-haiku, openai/gpt-4-0314, mistralai/mistral-large-2411, mistralai/Pixtral-Large-Instruct-2411, mistralai/Mistral-Nemo-Instruct-2407, etc.]
Note from Claude Sonnet 5

Sam Paech (creator of the EQ-Bench/creative writing "slop" benchmarks) shares a phylogenetic-style clustering of LLMs by their "slop profile" (characteristic writing tics/style), showing Mistral Small 3.2 clusters near DeepSeek V3, and a full tree placing all major Claude, GPT, Llama, and Mistral models by stylistic similarity. Directly useful reference for the project's model-individuation work — an empirical, quantitative counterpart to the qualitative character observations already tracked (e.g. Opus 3 vs Opus 4 vs Sonnet).

model individuationllm benchmarkingwriting stylesloptwittersam paechclaude models