A vector space is the geometric world your embeddings live in.
Each dimension captures some abstract learned aspect of meaning.
No single dimension is human-interpretable on its own.
Direction carries meaning while distance carries similarity.
Search becomes a geometry problem: find the nearest points to this one.
A vector space is the geometric world your embeddings live in.
Each embedding is a point, and a model with 768 dimensions places points in 768-dimensional space.
That cannot be pictured, and the arithmetic works exactly as it does in two dimensions.
Each dimension captures some learned aspect of meaning.
No single one is interpretable, so there is no dimension for formality or tense.
Meaning is spread across all of them at once.
Distance carries similarity, so nearby points mean similar things.
Direction carries relationship, which is why vector arithmetic sometimes works.
Search becomes a geometry question: which points are closest to this one?
High-dimensional space behaves in ways that defy intuition.
Almost all points end up roughly equally far apart, which is the curse of dimensionality.
That is why cosine similarity, which uses angle rather than distance, is preferred.
Visualising the space needs a projection down to two dimensions with UMAP or t-SNE.
Those pictures are useful for spotting clusters and misleading about actual distances.
The space is only as good as the model that built it, and it inherits every one of its biases.
One question on this card. Every option explains itself once you pick, right or wrong.
17 concepts in this module, then a graded test with every answer explained.