BlogData engineering

Hybrid search and reranking: getting retrieval right

Vector search finds passages that mean the same as the query and misses exact terms such as product codes and names. Keyword search does the reverse. Hybrid search runs both and fuses the two ranked lists, and a reranker then reorders the shortlist by reading each candidate against the query. Add them in that order, and measure each step on a labelled set of your own queries.

One query, two wrong answers

A travel insurer gives its claims handlers an assistant that searches the policy wordings. A handler asks: “Does TRV-2209 cover a missed connection?”

Vector search returns clauses about missed connections, and the top three come from three other policies. To an embedding model, TRV-2209 and TRV-2290 are almost the same string, and the meaning it can use is all in “missed connection”. The clause from the right policy is fourth.

Keyword search finds TRV-2209 at once. It puts the schedule page first, because that page repeats the code nine times, and the general exclusions second. The clause that answers the question is sixth. It is headed “Failure of onward travel arrangements” and never uses the word “missed” or the word “connection”.

If the assistant passes the top three results to the model, each method on its own produces a confident answer from the wrong text. The right passage was in both lists and at the top of neither.

The pattern is general. Identifiers, names, error codes and rare jargon belong to keyword search. Paraphrase, synonyms, questions worded unlike the document and queries in another language belong to vector search. Real users send both kinds, often in one sentence. Our RAG guide places this step in the whole pipeline. This article is about getting the step right.

BM25 in plain words

BM25 is the scoring formula behind most keyword search, and the default in Elasticsearch. It rests on three ideas.

  • A query word that appears in a passage counts in its favour, and each further appearance counts for less. The ninth “TRV-2209” adds almost nothing.
  • Rare words count for more than common ones. “TRV-2209” appears in a handful of documents and “cover” in nearly all of them.
  • Long passages are marked down a little, because they contain more words by chance.

Two parameters control the first and third: k1 for how quickly repeats stop counting, and b for the length adjustment. Elasticsearch’s defaults are 1.2 and 0.75, and we rarely change them.

What does need attention is the analyser, the step that splits text into terms. It decides whether “TRV-2209” stays one term or becomes “trv” and “2209”, whether words are stemmed, and which language rules apply. Most of the BM25 failures we debug are tokenisation failures. Test your identifier queries before anything else.

BM25 has no understanding. “Onward travel” and “connection” share no terms, so to BM25 they have nothing in common.

Fusing the two result lists

You now have two ranked lists whose scores cannot be compared. BM25 scores have no upper limit. Cosine similarities sit in a narrow band. Adding them together lets one side drown the other.

Reciprocal rank fusion

Reciprocal rank fusion (RRF) ignores the scores and uses positions. Each passage gets 1 / (k + rank) from every list it appears in, and the contributions are added. With the usual constant of 60, the travel query looks like this:

Passage Vector rank Keyword rank RRF score
TRV-2209, onward travel clause 4 6 1/64 + 1/66 = 0.0308
TRV-2209, general exclusions 30 2 1/90 + 1/62 = 0.0272
Other policy, missed connection clause 1 not listed 1/61 = 0.0164
TRV-2209, schedule page not listed 1 1/61 = 0.0164

The passage that was fourth and sixth is now first. Two moderate votes beat one strong one, which is the behaviour you want when each voter has a known blind spot.

RRF works acceptably with no tuning, which is why Elastic recommends it for hybrid search. Its RRF reference sets 60 as the default constant. Do not assume the constant, though. OpenSearch also defaults to 60, while Qdrant’s default is 2 with ranks counted from zero. The same two lists fuse differently in different engines, so set the value yourself.

Weighted scores

The alternative is to normalise each list’s scores to a common range and take a weighted sum. One number, often called alpha, sets the balance. In Weaviate, an alpha of 1 is pure vector search and 0 is pure keyword search, and score-based fusion has been the default method since version 1.24. OpenSearch offers score normalisation, and since version 2.19 rank-based fusion through its score-ranker processor, with optional weights for each side.

Score fusion keeps information that ranks throw away: a result far ahead of the rest stays far ahead. It is also sensitive to outliers, and the weight has to be tuned. Qdrant’s hybrid query documentation advises tuning weights only against an evaluation set and staying with plain RRF otherwise. We agree. Start with RRF, and move to weights when labelled data shows that one side deserves more say.

What does a reranker add?

First-stage retrieval embeds the query and the passages separately and compares vectors. That is fast, because passage vectors are computed ahead of time, and the model never sees query and passage together.

A reranker, usually a cross-encoder model, does see them together. It takes the query and one candidate, reads both, and outputs a relevance score. It can notice that a passage concerns the right policy and the right event, where an embedding only knows the passage is roughly about travel disruption. This is far too slow to run over a whole corpus and perfectly affordable over fifty candidates.

You have three ways to get one. Hosted APIs such as Cohere Rerank take a query and a list of documents and return them in order of relevance. Open-weight rerankers, such as the BGE and Qwen3 families, run on your own hardware. And some search platforms build one in: Azure AI Search runs its semantic ranker after fusion.

Some practical points:

  • Retrieve 50 to 100, keep 5 to 10. More candidates raise recall and latency together.
  • It is another model call on every query. Measure the added latency at your candidate count and decide whether your interface can carry it.
  • Scores are for ordering. Cohere’s guidance is that its relevance scores are relative, and that a cut-off should be found from borderline examples in your own domain. Do not assume 0.5 means “relevant”.
  • Check the input limit. Rerankers truncate or split long passages, and the limit differs between model versions.
  • It cannot repair recall. A reranker reorders what the first stage found. If the right passage is not among the candidates, it stays lost.

Query rewriting, chunk size and overlap

Fusion and reranking work on the query and the chunks they are given. Both can be improved before the search runs.

Query rewriting. In a conversation, the follow-up “and for the family plan?” is not searchable alone. A small model can rewrite it into a standalone query using the history. The same step can expand abbreviations and split a two-part question into two searches. The danger is that a rewrite paraphrases an identifier away. Pass codes and names through untouched, and consider sending the original text to the keyword side and the rewritten text to the vector side. Questions that need several searches in sequence are a different technique, covered in agentic RAG and GraphRAG.

Chunk size. The chunk is the unit search returns. Small chunks match precisely and lose their context. Large chunks keep context, blur the embedding and fill the prompt. We start with chunks that follow the document’s own structure, such as a clause, a section or a table with its header, at a few hundred tokens each.

Overlap. A sentence or two shared between neighbouring chunks stops a fact being cut in half at a boundary. More than that mostly produces duplicate results.

One habit helps both retrievers: prefix every chunk with its document title and section path. That prefix is what puts “TRV-2209” into every chunk of the policy for BM25, and tells the embedding what “this cover” refers to. Treat chunk size as a parameter to test, not a constant to copy from a tutorial.

How do you measure retrieval quality?

Build a labelled set: 50 to 200 real queries, each with the IDs of the passages that answer it. Include identifier queries, natural-language questions and a few with no answer in the corpus. Then three measures cover most needs.

Recall at k is the share of the relevant passages that appear in the top k. It answers “did we find it at all?” and suits the first stage, with k set to the number of candidates you pass to the reranker.

MRR, mean reciprocal rank, takes one divided by the position of the first relevant result and averages it over queries. First place scores 1, fourth place 0.25. It suits questions with one right passage.

nDCG at k gives credit for every relevant result in the top k, less the lower it sits, and divides by the score of the best possible ordering, so 1.0 is perfect. It copes with several relevant passages and with graded labels, such as 2 for “answers it” and 1 for “related”. It suits the final ranking.

Measure after each stage, and read the pattern. Recall at 50 should rise when you add the second retriever and stay flat when you add the reranker. nDCG at 10 should rise at both steps. If recall does not move after going hybrid, the keyword side is contributing nothing, and the analyser is the first suspect. Split the results by query type as well. A change that helps code lookups and hurts plain questions is a weighting problem.

What search engines offer today

Most engines now do the fusion for you. This is what the documentation showed when we checked.

Engine Hybrid search Worth knowing
Elasticsearch RRF over keyword and vector retrievers Rank constant 60 by default
OpenSearch Hybrid query with a search pipeline Score normalisation, or RRF since 2.19
Azure AI Search RRF for every hybrid query Semantic ranker can run after fusion
Qdrant Several prefetch searches, then fusion RRF or score-based fusion, plus multi-stage rescoring
Weaviate Hybrid query with an alpha weight Score-based fusion by default
Milvus Built-in BM25 beside dense vectors Text becomes sparse vectors on the server
Postgres Full-text search plus pgvector You write the fusion query

Built-in fusion is a convenience, not a reason to switch stores. How to choose one is in vector database or pgvector, and the index mechanics underneath are in vector search explained.

Where to start

In the first week, label fifty queries and record a vector-only baseline: recall at 50 and nDCG at 10. Add BM25 with RRF and measure. Add a reranker and measure. Keep each change only if the numbers move.

The first decision is whether your current store can do keyword search properly, with analysers you control. If it cannot, that matters more than which vector index it uses.

Retrieval tuned this way is the larger part of any RAG and enterprise search project we take on at WeStack AI, and it rests on the data engineering underneath.

Questions

Asked often.

What is hybrid search?

Hybrid search runs a keyword search, usually BM25, and a vector search for the same query, then merges the two ranked lists into one. Keyword search catches exact terms such as codes and names. Vector search catches passages that say the same thing in different words. Together they miss far less than either does alone.

What is reciprocal rank fusion?

Reciprocal rank fusion merges ranked lists using positions only. A document scores one divided by a constant plus its rank in each list, and the scores are added up. A document near the top of both lists wins. Because raw scores are ignored there is nothing to calibrate, which makes it a sound default.

Do I need a reranker if I already have hybrid search?

Usually yes, if answer quality matters. Fusion decides which candidates reach the shortlist. A reranker reads the query and each candidate together and puts the best few at the top, which is what the language model will see. Retrieve fifty to a hundred candidates and rerank down to five or ten.

What is the difference between recall at k, MRR and nDCG?

Recall at k asks whether the relevant passages appear anywhere in the top k results. MRR looks only at the position of the first relevant result. nDCG scores the whole ordering, giving more credit for relevant results placed higher and allowing graded relevance. Use recall for the first stage and nDCG for the final ranking.

Not on the list? Ask us directly

Keep reading

More notes from the build.

All blogs

Let’s build your stack.

Send a brief and we’ll reply with next steps.