Cosine, Dot Product and Euclidean in vector search

Module 08 · Embeddings & Search · 17 concepts

In depth / 15 lines

Cosine, Dot Product and Euclidean, in more detail

Every vector index asks you to pick a metric, and the choice matters less than people fear.

Cosine similarity is the cosine of the angle between two vectors, so it measures direction alone.

It runs from one for identical direction, through zero for unrelated, to minus one for opposite.

Because length is divided out, a long document and a short one about the same topic score alike.

Dot product multiplies the vectors element by element and sums, keeping both direction and length.

It is unbounded, and a vector with a larger magnitude will outscore a shorter one pointing the same way.

That can be a feature if length encodes something meaningful, such as confidence or popularity.

More often it is a bug, letting verbose documents dominate results for no good reason.

Euclidean distance is the ordinary straight-line gap between the two points.

It is a distance rather than a similarity, so nearer means smaller and the sort order is reversed.

The important fact is that on unit-length vectors all three produce the same ranking.

Cosine becomes the dot product, and squared Euclidean distance becomes two minus twice the dot product.

That monotone relationship means the neighbours come back in the same order whichever you choose.

This is why most embedding models return normalised vectors and most guides say the choice hardly matters.

It only starts to matter when your vectors are not normalised, and then dot product is the one to be careful with.

Deep dive

What is underneath Cosine, Dot Product and Euclidean

Three measures, and how they relate

d_{L2}(\mathbf{a}, \mathbf{b}) = \sqrt{\sum_{i=1}^{d} (a_i - b_i)^2}

Euclidean distance is the straight line between two points. It is small when the vectors are close in both direction and length.

\mathbf{a} \cdot \mathbf{b} = \sum_{i=1}^{d} a_i b_i = \|\mathbf{a}\| \|\mathbf{b}\| \cos\theta

The dot product grows with both the alignment and the magnitudes, so a long vector scores well against everything.

\cos(\mathbf{a}, \mathbf{b}) = \frac{\mathbf{a} \cdot \mathbf{b}}{\|\mathbf{a}\| \|\mathbf{b}\|}

Cosine similarity divides the magnitudes out and keeps only the angle, which is why it is the default for embeddings.

The identity that makes the choice mostly irrelevant

\|\mathbf{a} - \mathbf{b}\|^2 = \|\mathbf{a}\|^2 + \|\mathbf{b}\|^2 - 2\,\mathbf{a} \cdot \mathbf{b}

\|\mathbf{a}\| = \|\mathbf{b}\| = 1 \implies \|\mathbf{a} - \mathbf{b}\|^2 = 2 - 2\cos\theta

On normalised vectors, Euclidean distance is a decreasing function of cosine similarity, so both produce exactly the same ranking.

The dot product on normalised vectors is cosine similarity by definition, so all three agree once the lengths are 1.

That is why almost every embedding is normalised at write time: it makes the metric choice free, and it lets the index use the cheapest of the three.

When the magnitude carries meaning

On unnormalised vectors, dot product and cosine can rank results differently.

Some models encode confidence or term frequency in the norm, and normalising discards it.

A recommendation model may use length for popularity, so the dot product deliberately favours popular items and cosine deliberately does not.

The rule is to use the metric the model was trained with. A model trained with a contrastive loss on normalised vectors is being misused under raw Euclidean distance.

The others worth knowing

d_{L1} = \sum_{i=1}^{d} |a_i - b_i|

Manhattan distance sums absolute differences and is less sensitive to a single large deviation than L2, which squares it.

J(A, B) = \frac{|A \cap B|}{|A \cup B|}

Jaccard works on sets rather than vectors and is the right metric for sparse keyword or n-gram representations.

d_H = \sum_{i=1}^{d} \mathbf{1}[a_i \neq b_i]

Hamming distance counts differing bits, which is why binary quantised embeddings are compared with it and why that comparison is so fast.

One choice, in order

Read what loss the embedding model was trained with.
If it was trained on normalised vectors, normalise at write time and never again.
Pick the metric the index implements most efficiently, which is usually the inner product.
Confirm that the index and the query use the same metric, since a mismatch fails silently and returns plausible nonsense.
Measure recall against a brute-force scan before trusting any of it.

Above a few hundred dimensions every pair of random points sits at almost the same distance.

The absolute value of a similarity score therefore means very little, and only the ranking is worth reading.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1

How does dot product differ from cosine similarity?

Embeddings & Search

17 concepts in this module, then a graded test with every answer explained.

Study this module →