Start studying
Module 8 · Embeddings & Search Module contents
Module 8  ·  Card 1 / 17beginner

Embedding Models

An embedding model turns text into a fixed-length vector of numbers.

Similar meanings end up close together in that vector space.

Typical dimensions run from 384 up to 3072.

They are trained so that related pairs are pulled together and unrelated ones pushed apart.

Both your documents and your queries must use the same embedding model.

Learn in depth ↓
Module 08 · Embeddings & Search · 17 concepts
In depth  /  15 lines

Embedding Models, in more detail

An embedding model turns a piece of text into a fixed-length list of numbers.

The same length comes back whether you send three words or three paragraphs.

Those numbers are coordinates, and texts with similar meaning land near each other.

Typical sizes run from 384 dimensions up to 3,072.

Larger is not automatically better, and it costs more to store and to search.

They are trained on pairs known to be related, such as a question and its answer.

Related pairs are pulled together in the space and unrelated ones pushed apart.

That objective is why they beat older word vectors at sentence-level meaning.

The single most important rule is to embed queries and documents with the same model.

Two models produce incompatible spaces, and comparing across them returns noise.

Changing your embedding model means re-embedding your entire corpus.

That is an expensive migration, so the choice deserves a benchmark before you commit.

Most have an input limit of a few hundred to a few thousand tokens.

Text beyond it is silently truncated, which quietly ruins retrieval for long documents.

Domain-specific models exist for code, law and medicine and usually beat general ones on their turf.

Want the full technical article? Wikipedia ↗
Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
Why must documents and queries use the same embedding model?
Embeddings & Search

17 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close