pca

Lari Island ✓ @Lari_island · 24m Green - Opus 3 Orange - Opus 4 Blue - Fable 5 Note the Opus 4's range The horizontal axis is PC0 of the whole model space across labs [Chart: 2D scatter plot, dark background, no axis tick labels beyond an implied PC0 horizontal axis and unlabeled vertical axis. Points colored teal/blue (left cluster), orange (right cluster spreading both upper-right and lower-right), one small green dot near center-right, and one blue dot at bottom center. A small text annotation near top center reads "-PC2 / am / cathedral" (partially legible).] > QUOTED: Lari Island ✓ @Lari_island · May 23: Opus 3 (blue) and all other Opuses in a PCA space calculated for model variation (not item variation) across 79 models - in a "model 3D space." [with small embedded thumbnail of an earlier, differently-colored version of the plot]

Alex Clemmer 🔥🔥...✓ @haus... · Mar 28 The first time you hear about the JL lemma, it will seem too good to be true. And it is, kind of, I'll explain. The idea is: if you have points in large d-dimensional space, a RANDOM projection to much smaller k-dim subspace will be "nearly optimal" "in the general case." Or, more specifically: with high probability, the pairwise distances between points are preserved, given a couple other requirements around d and k. So why don't we just use random projections instead of carefully-constructed ones all the time? This is the most common misunderstanding of the JL lemma, and the one thing to really understand about it: in many (most?) datasets that are meaningful to humans, you actually CAN do better with something like maybe PCA. If your dataset is pathological, e.g., the points all lie on a plane even though it's technically in 3 dimensions, then clearly some planes you project onto will be better than others. The JL lemma does not apply to 2 and 3 dimensions, but you can imagine this would be true in large numbers of dimensions too. (See screenshot 1, i hope you like it because i made it myself lol.) If you know just those facts, you will be pretty well-prepared to answer most questions about its use. Most of the papers Delip mentions do presuppose that you know this. At least when I was a student, I found this to be non-obvious. [Embedded diagram: two 3D cube diagrams labeled "fig. 1: randomly projecting from R^3 to R^2" — left "random projection is nearly optimal", right "PCA is far better than random projection", each showing points projected onto a 2D plane inside a cube] > QUOTED: Delip Rao e/σ ✓ @deliprao · Mar 27 > The Google turboquant paper is [cut off]