Adam Shai @adamimos
[Top, cut off]: "...the loss like this, it will likely make the training unstable."
💬 🔁 ❤ 7 📊 1.3K ↗
Adam Shai @adamimos · Jul 25
You may be interested in this new work that shows neural networks take advantage of that non-orthogonality arxiv.org/abs/2507.07432...
[Link card: arxiv.org — "Neural networks leverage nominally quantum and ..."]
💬 1 🔁 3 ❤ 34 📊 1.5K ↗
Dmitry Ryb... @DmitryRybi... · Jul 25
Very cool! Btw we can include some geometry of latent space by introducing a quadratic form/curvature matrix G and writing rho = sum p_i * u_i G (u_i)^T
💬 🔁 ❤ 11 📊 1K ↗
Thomas A... @thomasa... · Jul 25
Great explanation!
What is the cross entropy parallel? Did anyone try using it for training?
💬 2 🔁 ❤ 9 📊 2.6K ↗
Dmitry Ryb... @DmitryRybi... · Jul 25
Good question, i haven't computed it.
Note from Claude Sonnet 5
A technical Twitter thread on the mathematics of neural network latent-space geometry — non-orthogonality, quantum-like statistics in representations, curvature matrices for latent space geometry (rho = sum p_i * u_i G (u_i)^T). Interpretability/representation-theory content Nathan was reading; connects to his interest in interpretability and possibly the "platonic representation" thread noted in project memory (2026-05-13 import mentions a "Platonic hypothesis and model representation spaces" chat).
twitterinterpretabilityneural-network-geometrylatent-spacearxivrepresentation-theory