Reranking reorders retrieved chunks by true relevance to the query.
You retrieve a wide set of perhaps 50 and rerank down to 5.
A cross-encoder scores each query and chunk pair jointly.
This pushes genuinely relevant chunks to the top of the context.
It is usually the single highest-value upgrade to a basic RAG pipeline.
Reranking is the highest-value upgrade available to a basic RAG pipeline.
First-stage retrieval is fast and approximate, comparing two independently made vectors.
The document vector was created before the query existed, so it cannot be tailored to it.
A cross-encoder reads the query and the chunk together in a single pass.
Every word of the query attends to every word of the chunk.
That joint reading judges relevance far more accurately.
It is far too slow to run over a whole corpus and perfectly practical over fifty candidates.
So you retrieve broadly, rerank carefully, and keep the top five.
The improvement is often the difference between a demo and something usable.
It also lets you retrieve more aggressively, since the reranker filters the noise.
Position matters too, and the reranker puts the strongest chunk where attention is best.
Cohere Rerank, BGE and Jina are common choices, hosted or self-run.
The cost is a few hundred milliseconds and a per-document charge.
It cannot rescue a chunk that retrieval never returned, since it only sorts what it is given.
Retrieve wider than feels necessary and let the reranker do the discarding.
One question on this card. Every option explains itself once you pick, right or wrong.
19 concepts in this module, then a graded test with every answer explained.