Start studying
Module 4 · NLP Module contents
Module 4  ·  Card 13 / 20intermediate

Cosine Similarity

Cosine similarity measures the angle between two vectors.

It ranges from -1 for opposite to 1 for identical direction.

It ignores vector length and looks only at direction.

That makes it robust to differences in document length.

It is the standard similarity measure for embeddings and vector search.

Learn in depth ↓
Module 04 · NLP · 20 concepts
In depth  /  15 lines

Cosine Similarity, in more detail

Cosine similarity measures how close two vectors point, ignoring how long they are.

It is the cosine of the angle between them.

Pointing the same way gives one, at right angles gives zero, opposite gives minus one.

In practice with text embeddings the values sit between about zero and one.

It is computed by multiplying the two vectors element by element and dividing by their lengths.

That division is what removes length from the comparison.

Length matters in text, because a long document repeats its words more often.

Under Euclidean distance a long and a short document on the same topic look far apart.

Under cosine they look almost identical, which is the answer you wanted.

This is why cosine is the default for embeddings and for vector search.

If your vectors are already normalised to length one, cosine and dot product are the same thing.

Most embedding APIs normalise for you, so the dot product is used for speed.

It is cheap enough to run against millions of vectors, which is what makes search practical.

It tells you nothing about why two things are similar.

And it inherits every bias and blind spot of the embedding model that produced the vectors.

Want the full technical article? Wikipedia ↗
Deep dive

What is underneath Cosine Similarity

The formula

cos(a,b)=a⋅b∥a∥∥b∥=∑i=1daibi∑iai2∑ibi2\mathrm{cos}(\mathbf{a}, \mathbf{b}) = \frac{\mathbf{a} \cdot \mathbf{b}}{\left\| \mathbf{a} \right\| \left\| \mathbf{b} \right\|} = \frac{\sum_{i=1}^{d} a_i b_i}{\sqrt{\sum_i a_i^2} \sqrt{\sum_i b_i^2}}

The numerator is the dot product. The denominator is the product of the two vectors' lengths.

The result is the cosine of the angle between them, which is bounded.

−1≤cos(a,b)≤1-1 \leq \mathrm{cos}(\mathbf{a}, \mathbf{b}) \leq 1

One means the same direction, zero means perpendicular, minus one means opposite.

What dividing by the norms buys

Without it you have the raw dot product, which grows with the length of either vector.

a⋅b=∥a∥∥b∥cos⁡θ\mathbf{a} \cdot \mathbf{b} = \left\| \mathbf{a} \right\| \left\| \mathbf{b} \right\| \cos\theta

For text, length usually encodes something you do not want: a long document has a bigger vector than a short one on the same topic, purely because more words were counted.

Normalising throws that magnitude away and keeps only the direction, so a one-paragraph and a ten-page treatment of the same subject come out similar.

cos(a,b)=cos(ka,b)for any k>0\mathrm{cos}(\mathbf{a}, \mathbf{b}) = \mathrm{cos}(k\mathbf{a}, \mathbf{b}) \quad \text{for any } k > 0

Its relationship to Euclidean distance

They are not independent. On vectors that have already been normalised to unit length, one determines the other.

∥a−b∥2=2−2 cos(a,b)\left\| \mathbf{a} - \mathbf{b} \right\|^2 = 2 - 2 \, \mathrm{cos}(\mathbf{a}, \mathbf{b})

So ranking by cosine similarity descending and by Euclidean distance ascending give the identical order, provided the vectors are unit length.

That is why embedding stores normalise on insert: it lets them use a plain dot product, which is a single fused instruction, and get cosine ordering for free.

One comparison, in order

  1. Compute the dot product of the two vectors.
  2. Compute each vector's Euclidean norm.
  3. Divide the dot product by the product of the norms.
  4. Guard against a zero norm, which is undefined rather than zero.
  5. Or normalise both vectors once, in advance, and take the dot product alone.

Where it is the wrong choice

If magnitude carries meaning, discarding it is a loss.

Count vectors where the total matters, and embeddings whose norm encodes confidence, are both cases where the plain dot product is the right metric.

It is also not a distance in the mathematical sense, because it does not satisfy the triangle inequality.

An index built on it therefore needs a metric conversion first.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
Why is cosine similarity preferred over raw distance for embeddings?
NLP

20 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close