Indexing stores the vectors in a structure built for fast similarity search.
Metadata such as source, date and permissions is stored with each chunk.
That metadata enables filtered search and access control.
The index has to support incremental updates as documents change.
Stale indexes are one of the most common causes of wrong RAG answers.
Indexing stores the vectors in a structure built for fast similarity search.
Without it, every query would compare against every chunk in the corpus.
Metadata is stored with each chunk and matters as much as the vector.
Source, title, date, author, document type and permissions are the usual fields.
Metadata is what makes filtered search possible.
Restricting to documents from this year, or to those this user may read, happens here.
Access control belongs in the index, since it cannot be added convincingly in the prompt.
A model told to ignore a document it can see is not a security boundary.
The index must support incremental updates, because documents change constantly.
Rebuilding from scratch nightly works until the corpus grows past a certain size.
Deletions are the awkward case, and many approximate indexes only mark rather than remove.
Stale indexes are among the most common causes of confidently wrong RAG answers.
The retrieved text was true when indexed and is no longer true now.
Store the indexed date so answers can be qualified or filtered by age.
Track index size, update lag and failed documents as ordinary production metrics.
One question on this card. Every option explains itself once you pick, right or wrong.
19 concepts in this module, then a graded test with every answer explained.