Chapter 7 · concept 3 of 6 · show all

Hybrid retrieval and rank fusion

Run keyword and vector search, then merge the two ranked lists with Reciprocal Rank Fusion. On our corpus fusion kept all 47 answers in the top 3 and fixed every vector miss.

A real question

Keyword search and vector search give two different lists. How do you combine them?

Give every chunk points for its rank in each list and add them up. That is Reciprocal Rank Fusion. A chunk near the top of either list rises; one near the top of both rises most.

The problem

Why this matters

The belief

Just add the BM25 score and the cosine score.

What really happens

Our top BM25 scores ranged from 5.1 to 24.9; cosine from about 0.4 to 0.85. Adding them lets BM25 drown out the vector. RRF uses only ranks, so the scales never need to match.

Why not the simpler way

RAG-27 showed BM25 and vectors fail on different questions. Picking one means accepting its blind spot. Running both is easy; the hard part is merging two lists whose scores mean different things.

The idea

The idea in plain words

Reciprocal Rank Fusion (Cormack, Clarke and Büttcher, SIGIR 2009) scores each item by sum over lists of 1 / (k + rank). It ignores the raw scores. The constant k (usually 60) controls how much the very top ranks dominate.

Two friends recommend restaurants. Rather than comparing one friend's "8 out of 10" with the other's "four stars", you favour the places near the top of both lists.

How it works

Step by step

1
Run both searches
Take the top 10 from BM25 and the top 10 from vector search for the same question.
2
Points by rank
Each chunk gets 1 / (60 + rank) from each list it appears in. Rank 1 is worth 0.0164, rank 10 is worth 0.0143.
3
Add up and sort
Chunks in both lists collect points twice. Sort by the total and keep the top few.

In detail

Inputs. The top N from each retriever, often 10 to 50. Items missing from a list get nothing from it.

k. Small k makes rank 1 count far more than rank 5; large k flattens the curve. On our corpus k = 1, 10, 60 and 100 all kept 47 of 47 in the top 3.

Weighted and score-based fusion. You can weight lists (for example BM25 twice), or normalize each list's scores to 0 to 1 and add them. Our min-max score sum also found 47 of 47. Score methods need re-tuning when a model changes; RRF does not.

After fusion comes the reranker (RAG-31), which reads the fused candidates closely.

Fusion uses only each chunk's position in the two lists, so a BM25 score of 9.7 and a cosine of 0.67 never need to be compared.

The code

Try it in Python

def rrf(lists: list[list[str]], k: int = 60) -> list[tuple[str, float]]:
    """Reciprocal Rank Fusion: each list is chunk ids, best first."""
    scores: dict[str, float] = {}
    for ranked in lists:
        for rank, cid in enumerate(ranked, start=1):
            scores[cid] = scores.get(cid, 0.0) + 1.0 / (k + rank)
    return sorted(scores.items(), key=lambda x: -x[1])

fused = rrf([bm25_top10, vector_top10])[:3]

See it

What we measured

Measured · retrieval on 47 golden questions

Polder corpus, 34 recursive 512 token chunks. BM25 and bge-small vector search, top 10 each, fused with RRF.

Answer in top 3vector 44 · BM25 47fused 47

Answer ranked firstvector 30 · BM25 35fused 33

Mean reciprocal rankvector 0.784 · BM25 0.862fused 0.840

RRF k = 1, 10, 60, 10047 in top 3 for every k

Min-max score sum instead of RRF47 in top 3

Measured 2026-11-04 with corpus/scripts/measure-parts-4-7.mts. On this corpus BM25 alone was slightly better at rank 1; fusion is the safer choice because it does not depend on which retriever suits the question.

Try it on a real example

Two questions, rank of the answer chunk

Question

“How much notice is given for a price change during a fixed term?”

BM25: rank 1 · vector: rank 4 · fused: rank 1

Vector alone missed the top 3. Fusion keeps BM25's find on top.

Loading 34 chunks…

Choosing

Use it when, avoid it when

Use it when

Hybrid search with RRF as the default retriever
Logging both ranks to debug misses

Avoid it when

Adding raw BM25 and cosine scores
Tuning fusion weights before you have a golden set

What it costs · Two searches instead of one, run in parallel. No model calls.

In production

Hybrid search roughly doubles retrieval work, but both searches are fast compared with the model call, and they can run in parallel. Many search engines and vector stores offer it built in. Log both ranks for every query: when an answer is missing, the two ranks tell you which retriever lost it (RAG-32).

Watch out

Common mistakes

Fusion always beats both inputs. BM25 alone ranked more answers first here.
k must be tuned. 60 is a safe start. Our 34 chunks were too few to show a difference, so tune k only if a bigger golden set does.
Hybrid needs a special database. The fusion step is about ten lines of code, but you do run and keep up a second (keyword) index next to the vector one.

Check yourself

Remember this

Remember these

Hybrid search runs BM25 and vector search on every question.
RRF merges by rank: score = sum of 1 / (k + rank), with k = 60.
Fusion put all 47 answers in the top 3, fixing the 3 vector misses. BM25 alone also had 47, so this small corpus cannot show fusion beating it.
k from 1 to 100 gave the same top 3 here, but with 34 chunks that test is weak; start at 60 and re-check on a bigger set.

Say it in an interview

01I run BM25 and vector search and merge them with Reciprocal Rank Fusion.
02RRF uses ranks, so cosine and BM25 scales never need to match.
03BM25 alone ranked slightly more answers first; fusion wins on robustness.
04I log both ranks per query so a miss points to the retriever that lost it.

Try it yourself: Retrieval LabKeyword, vector and hybrid search side by side, with the rank of the answer in each.

Quick quiz

Three questions

Why does RRF use ranks instead of the raw scores?

On our corpus, which got more answers ranked first?

Q3 · from RAG-27

Which kind of question does BM25 handle better than vector search?

Saved on this device. No account needed.