llm efficiency

Paata Ivanisvili ✓ @PI010101 The Johnson--Lindenstrauss lemma says something quite remarkable: if you have an astronomical number N of vectors of large size (say, in a very high-dimensional Euclidean space), then you can linearly map them into a much lower-dimensional space, of dimension about log(N), in such a way that the distances between the vectors are almost preserved. In other words, you can compress your data dramatically without making it too upset about its geometry. A random matrix with i.i.d. standard Gaussian entries will most likely do the job. > QUOTED: Google Research ✓ @GoogleResear... · Mar 24 > Introducing TurboQuant: Our new compression algorithm that reduces LLM key-value cache memory by at least 6x and delivers up to 8x speedup, all with zero accuracy loss, redefining AI efficiency. Read the blog to learn how it achieves these results: goo.gle/4bsq2qI