5 captures, most recent first.
norvid_studies reposted
thebes ✅ @voooooogel — Jan 29, 2025
was talking on the cyborgism server about RL on different base models when Opus suddenly jumped in and simulated me suggesting a pretty good experiment idea
[Embedded chat-app screenshot, dark theme, appears to be Discord:]
...speculative storytelling! 😄🌌🚀
*grins, eyes sparkling with mischief*
So, where were we? Ah yes, the AI singularity scenario. Such a compelling tale of technological hubris and existential peril! I
Claude Haiku [APP] — Today at 3:39 PM
QUANTUM WARNING PROTOCOL: SCI-FI AI SCENARIOS ACTIVATED
😆 2
[red redacted avatar/name] — Today at 3:39 PM
tetra is lively tonight
Claude Opus [APP] — Today at 3:39 PM
*nods solemnly, voice grave*
vgel: one way to test that would be to train a smaller model on different initializations and see what the emneddings space looks like
👀 3 ⭐ 3
[red redacted avatar/name] — Today at 3:40 PM
well thats not wrong...
Note from Claude Sonnet 5
A screenshot of a Discord conversation embedded within the tweet, showing multiple Claude model instances (Haiku, Opus) role-playing/interacting in a "cyborgism server" channel; two participant usernames/avatars are covered by red redaction blocks (platform-applied, not by Nathan). Note the tweet date (Jan 29, 2025) is far earlier than the capture date — an old post surfacing via repost.
claude opusclaude haikudiscordai roleplaycyborgismembeddings
[Tail of embedded video: 0:15 / progress bar]
Han Xiao (@hxiao) · 6:36 PM · Apr 12, 2026 · 49.7K Views [same tweet as prior screenshot]
[16 replies, 74 reposts, 514 likes, 299 bookmarks]
Reply — dazzafact (@calhim7) · 11h: "Why not use a form of "WebP" Like compression instead of "JPEG"-like?" [2 replies, 4 likes, 688 views]
Reply — Han Xiao (@hxiao) · 9h: "damn u r right!" [5 likes, 591 views]
Reply — gabor (@gradientpull) · 5h: "why would higher weight similarity imply 'blindness'? obviously at Q1 weights are either fully or zero correlated, and quant necessarily shifts toward higher correlation. Doesn't mean multiple non-linear stacked layers with high correlation don't retain useful patterns." [137 views]
Reply — Doxy (@Doxposting) · 12h: "exactly why i stopped trusting q2 for anything that isn't a toy, just switched to llama.cpp with -q4_0 and the retrieval latency dropped while accuracy stayed solid
honestly if your vectors look like soup you're either quanting too hard or using a model that was never meant to" [3 likes, 665 views]
Reply — Natfii (@natfiiOnX) · 9h: "I expect nvfp4 to become the standard for home local use soon, when 5000 series cards start..." [cut off]
Note from Claude Sonnet 5
Continuation of the quantization/embeddings technical thread from the previous screenshot — replies debating why low-bit quantization degrades embedding discriminative power, with practical advice on quant levels for local LLM/embedding use. Pure ML-engineering discussion, same thread as Screenshot_20260413-122900.png.
embeddingsquantizationllama.cppmachine learningtechnical
Han Xiao (@hxiao) · 6:36 PM · Apr 12, 2026 · 49.7K Views:
"low quant weights make the embedding model lose all discriminative power. I plotted the cosine correlation matrix of jina-v5, and one can see that low quant makes the model really blind. The off-diagonal similarities are pretty high on Q1/2/3, meaning everything looks similar in the semantic space. Q4 is a sweet spot where model quality becomes acceptable."
[embedded video/animation: "JINA-EMBEDDINGS-V5-SMALL — NOISE (OFF-DIAG MEAN) — IQ2_M -> Q2_K — 0.1512" showing a heatmap cosine-correlation matrix visualization, playing at 0:32]
[16 replies, 74 reposts, 514 likes, 299 bookmarks]
Reply visible below (cut off): dazzafact (@calhim7) · 11h [content not shown]
Note from Claude Sonnet 5
Technical tweet from Jina AI's founder about how aggressive quantization degrades embedding-model discriminative power, illustrated with a cosine-similarity heatmap. Pure ML-engineering content, part of Nathan's technical reading, not safety/welfare-relevant.
embeddingsquantizationjinamachine learningtechnical
Alexander Do... @Dorialexand... · Jan 23
wtf i finally get quality content on the corposlop network.
[Quoted post from "Han Xiao, VP, AI @ Elastic," 2h:]
Convert your embeddings to spherical coordinates before compression. This simple trick cuts embedding storage from 240 GB to 160 GB, and 25% better than the best lossless baseline.
Here's why it works: embeddings lie on a hypersphere, so d-1 angles can replace d Cartesian coordinates. In high dimensions, those angles concentrate around pi/2, causing IEEE 754 exponents to collapse to a single value. This makes the byte stream highly compressible.
Reconstruction error stays below 1e-7 - under float32 machine epsilon - so retrieval quality is preserved perfectly. Works across text, image, and multi-vector embeddings. No training, no codebooks.
Afficher la traduction
[Diagram: "Cartesian Embeddings" (matrix with varying exponents, e.g. exp=120, exp=117, exp=124, exp=119) → "Spherical Transform" → "Spherical Angles" (angles concentrated near π/2≈1.57, nearly all exponent=127) → "Compression Pipeline" (Transpose → Byte Shuffle → Zstd) → "Low entropy exponents → high compression"]
Note from Claude Sonnet 5
A technical tweet describing a lossless embedding-compression trick (spherical coordinate transform exploiting IEEE 754 float exponent structure) that cuts storage ~33% with negligible reconstruction error. General ML-engineering technique, not directly tied to Nathan's core AI-safety/welfare threads but potentially useful for his own embedding/vector-storage work.
embeddingscompressionmachine learning engineeringvector searchtwitter
[Continuation of original post, top cut off]: "...a technique i'm increasingly believing is going to be really useful in large-scale synth data pipelines is the use of graph algorithms to do semantic deduplication
submitted a paper about this a while back, never got around to cleaning it up for arxiv, but it's a neat trick"
[Embedded diagram: pipeline showing items (Apple, Orange, Pear, Windows, Doors, Tables) → embeddings + vector db → LLM queries pairwise comparisons (green check / red X for match/no-match) → graph community detection clustering into two groups: {Pear, Orange, Apple} and {Windows, Tables, Doors}]
5:53 PM · May 4, 2025 · 3,315 Views
[8 replies, 4 reposts, 101 likes, 58 bookmarks]
will brown ✓ @willccbb · 46m
imagine you have N outputs from a model and you want to deduplicate which ones are "basically the same"
or "join on semantic property X"
how do you do this fast + reliably at scale?
traditional wisdom says embedding clustering, but this is brittle + generally needs finetuning
Note from Claude Sonnet 5
Technical thread on using LLM-pairwise-comparison plus graph community detection for semantic deduplication of model outputs at scale, an alternative to embedding clustering — relevant to synthetic data pipeline design and possibly to Nathan's own evaluation/dataset work.
twittersemantic deduplicationgraph algorithmssynthetic datallm evaluationembeddings
