Hybrid Search, explained in 5 lines

Module 08 · Embeddings & Search · 17 concepts

In depth / 15 lines

Hybrid Search, in more detail

Hybrid search runs keyword and vector retrieval together and merges the results.

It exists because the two methods fail in opposite ways.

Dense retrieval understands meaning and fumbles exact identifiers.

Sparse retrieval nails exact identifiers and misses synonyms entirely.

Running both means a query is covered whichever kind it turns out to be.

Real queries are frequently both, mixing a description with a product code.

The difficulty is combining the results, since the two scores are not comparable.

A BM25 score of 12.4 and a cosine score of 0.83 are on different scales entirely.

Normalising them is fragile, since the ranges shift with the query.

Reciprocal rank fusion sidesteps it by using only the positions in each list.

A document near the top of either list scores well, and one high in both scores best.

Weighting can favour one retriever, and equal weighting is a sound starting point.

Hybrid reliably beats either method alone on real corpora, which is why it is the default.

The cost is two indexes to build, keep in step and pay for.

Adding a reranker on top of hybrid results is the usual next improvement.

Want the full technical article? Wikipedia ↗

Deep dive

What is underneath Hybrid Search

Two systems that fail differently

A keyword index is perfect on a part number and blind to a paraphrase. An embedding index is the reverse, and neither weakness is fixable within its own family.

\text{recall}_{\mathrm{hybrid}} > \max(\text{recall}_{\mathrm{BM25}}, \; \text{recall}_{\mathrm{dense}})

Running both and merging is worth several points of recall on almost every benchmark, and the gain is largest exactly where each one alone is weakest.

The scores are not comparable

s_{\mathrm{BM25}} \in [0, \infty), \qquad s_{\mathrm{cos}} \in [-1, 1]

BM25 returns an unbounded sum that depends on query length and corpus statistics. Cosine similarity returns a bounded angle.

Adding them directly lets whichever has the larger numbers decide everything, and that is usually BM25 on a long query.

s' = \frac{s - \min}{\max - \min}

Min-max normalisation within each result list makes them comparable, and it is unstable: the scale depends on the best and worst result of that particular query.

s' = \frac{s - \mu}{\sigma}

Standardising is steadier but still assumes the two score distributions have a shape worth comparing, which they do not.

Weighted fusion

s = \alpha \, s'_{\mathrm{dense}} + (1 - \alpha) \, s'_{\mathrm{sparse}}

With normalised scores, one weight controls the balance, and $\alpha$ near 0.5 is the usual starting point.

The weight is corpus-specific. Technical documentation with many identifiers wants more sparse, and conversational content wants more dense.

It has to be tuned on labelled queries, and it has to be retuned whenever the embedding model changes.

Rank fusion, which avoids the problem

\mathrm{RRF}(d) = \sum_{r \in R} \frac{1}{k + \mathrm{rank}_r(d)}

Reciprocal rank fusion ignores the scores entirely and uses only the positions, so nothing needs normalising and nothing needs tuning.

It is the default in most hybrid systems for that reason: one constant, no calibration, and results that are usually close to a carefully tuned weighted sum.

One hybrid query, in order

Run the query against the inverted index and take the top $k$ .
Embed the query and search the vector index for its own top $k$ .
Merge the two lists by rank fusion, or by weighted normalised scores.
Deduplicate, since the same passage usually appears in both.
Rerank the merged top candidates with a cross-encoder.

k_{\mathrm{retrieve}} \gg k_{\mathrm{final}}

Both retrievers are asked for more than the final answer needs, because fusion can only reorder what it was given and a passage missing from both lists is lost.

The two searches are independent, so they run in parallel and the added latency is the slower of the two rather than their sum.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1

Why does hybrid search outperform dense retrieval alone?

Embeddings & Search

17 concepts in this module, then a graded test with every answer explained.

Study this module →