loss-functions

2 captures, most recent first.

2025-07-09

[top, cut off] "...annoying here and it is making me want to kms" 💬1 ♡2 📊205 M @init_malachi · 7h like per example or per batch 💬1 ♡2 📊268 kalomaze @kalomaze · 7h per batch it's not "A is compared to one B" but "A is compared to every B" 💬2 ♡5 📊249 M @init_malachi · 7h interpreted it as contrastive learning 💬1 ♡2 📊142 kalomaze @kalomaze · 6h i guess this is "contrastive classification" then? 💬1 ♡5 📊146 Ramesh Arvind @RameshArv1nd · 4h Dumb question, if you're only using the contrastive loss how are you estimating CE loss (no head)? And also why abandon CE and not add the contrastive term as an aux loss. I imagine for binary you could get away with some min/max sigmoidal diff across the batch [cut off]

Note from Claude Sonnet 5

Continuation of the same ML training-technique thread as the prior screenshot (kalomaze discussing pairwise/contrastive classification loss formulation). Technical ML discussion, not AI-safety focused.

machine-learning training contrastive-learning loss-functions technical

kalomaze @kalomaze

2025-07-09

kalomaze @kalomaze ok so even if your binary classification data isn't paired or pairwise and just belongs to two arbritary classes you can reduce cross entropy faster and better by formulating it as a pairwise thing across the batch >`Cross-entropy loss (not used for training): 0.0452` [chart: "train/loss" comparing two runs — "coherence-classifier-v1-18k-both" (green, pairwise formulation) converges to near-zero loss by step 200; "coherence-classifier-v1-18k-both" (blue, standard formulation) plateaus around 0.15-0.2 with high variance through step 500] 1:56 PM · Jul 9, 2025 · 4,632 Views

Note from Claude Sonnet 5

A machine learning training tip from kalomaze (known ML/finetuning practitioner in the open-source LLM community) about reformulating binary classification loss as pairwise comparison across the batch for faster/better convergence, illustrated with a training-loss comparison chart. Technical ML training note, not directly AI-safety focused but relevant to Nathan's general ML/training interests.

machine-learning training loss-functions classifier technical