Skip to main content

Combining Sparse and Dense: Fusion Strategies

Hybrid retrieval fusion combines rankings from BM25 (sparse) and dense embedding retrieval into a single ranked list. The core challenge is that BM25 and dense similarity scores are on different scales: BM25 scores might range from 0 to 50+, while cosine similarity ranges from -1 to 1. Naive score averaging will be dominated by BM25's larger magnitudes, burying valuable dense-retrieval results. Fusion strategies address this by either normalizing scores to comparable scales (min-max, z-score normalization) or using rank-based fusion algorithms that ignore absolute scores and combine rankings by position. The three most practical approaches are reciprocal rank fusion (RRF, parameter-free and robust), weighted normalization (learnable but requiring tuning), and learned fusion networks (state-of-the-art but complex). For most RAG applications, RRF is recommended: it is simple, requires no training, and achieves results within 2–3% of learned fusion on established benchmarks.

The Score Normalization Problem​

BM25 and dense retrieval operate in different score spaces. When you retrieve top-50 documents from each system, their scores cannot be directly compared:

  • BM25: Top document scores [35.2, 28.1, 22.4, ...]
  • Dense (cosine similarity): Top document scores [0.89, 0.85, 0.81, ...]

If you naively combine scores by averaging (or summing), the BM25 scores dominate: (35.2 + 0.89) / 2 = 18.0 vs (28.1 + 0.81) / 2 = 14.5. The dense retrieval's contribution is nearly invisible. To address this, you normalize both to a common scale before fusion.

Normalization Strategies​

Min-Max Normalization: Rescale each retrieval method's scores to [0, 1]:

normalized_score = (score - min_score) / (max_score - min_score)

For the example above:

  • BM25 normalized: [(35.2 - 22.4) / (35.2 - 22.4) = 1.0, (28.1 - 22.4) / 12.8 = 0.45, ...]
  • Dense normalized: [(0.89 - 0.81) / (0.89 - 0.81) = 1.0, (0.85 - 0.81) / 0.08 = 0.5, ...]

Combined (average): [(1.0 + 1.0) / 2 = 1.0, (0.45 + 0.5) / 2 = 0.475, ...]

Now both methods contribute equally. Min-max normalization is simple but assumes the observed min and max are representative. If your top-50 dense results happen to be clustered (small max-min range), normalization amplifies noise.

Z-Score Normalization: Rescale using mean and standard deviation:

normalized_score = (score - mean_score) / std_score

This is more robust to outliers and assumes scores are roughly normally distributed. For skewed distributions (many low scores, a few high scores), z-score can produce negative normalized scores, which you must clip to [0, 1].

Percentage Rank: Convert scores to percentile ranks (0–100), making all methods ordinal:

percentile_rank = (count of scores below this score) / (total scores) * 100

This discards absolute score information but ensures all methods are on the same scale. It is particularly useful if you do not trust the absolute magnitude of either score.

Reciprocal Rank Fusion (RRF)​

Reciprocal Rank Fusion is a parameter-free rank-based fusion algorithm that sidesteps score normalization entirely. For each document, it assigns a fusion score based on its rank position (not absolute score) in each ranked list:

RRF(d) = sum over all ranking methods m: 1 / (k + rank_m(d))

where k is a constant (typically 60) and rank_m(d) is the 1-indexed position of document d in method m's ranking. If a document does not appear in method m's top-k, it gets a score of 0 from that method.

Example: Consider a document that ranks #3 in BM25 and #7 in dense retrieval:

RRF = 1 / (60 + 3) + 1 / (60 + 7) = 1/63 + 1/67 ≈ 0.016 + 0.015 = 0.031

A document ranking #1 in both methods scores:

RRF = 1 / (60 + 1) + 1 / (60 + 1) = 1/61 + 1/61 ≈ 0.033

The constant k=60 dampens the influence of early-rank positions (small rank differences do not amplify), making RRF robust to outliers. Smaller k (e.g., 10) makes rank position matter more; larger k (e.g., 100) makes all ranks more similar.

Advantages:

  • Parameter-free: No training or tuning needed.
  • Robust: Works even if one retrieval method is much better or much worse than the other.
  • Interpretable: You can trace which retrieval method contributed most to the final ranking.

Disadvantages:

  • Rank-only: Ignores score magnitude, potentially losing signal. A document at rank #5 with score 0.99 is treated the same as rank #5 with score 0.51.
  • Fixed k: The constant k=60 is a heuristic; different corpora might benefit from different k values (typically 50–100).

Weighted Normalization Fusion​

If you have labeled data (queries with annotated relevant documents), you can learn optimal weights for combining the two methods:

hybrid_score(d) = w_bm25 * normalized_bm25(d) + w_dense * normalized_dense(d)

where w_bm25 + w_dense = 1. You optimize these weights to maximize a metric like Mean Average Precision (MAP) or Normalized Discounted Cumulative Gain (NDCG) on your labeled set.

For example, if you have 100 queries with relevance judgments:

from sklearn.linear_model import LogisticRegression
import numpy as np

# Prepare training data
X = []  # Features: [normalized_bm25, normalized_dense]
y = []  # Labels: 1 if relevant, 0 otherwise

for query in queries:
    bm25_results = bm25_retrieve(query)
    dense_results = dense_retrieve(query)
    
    # Normalize scores
    norm_bm25 = min_max_normalize(bm25_results)
    norm_dense = min_max_normalize(dense_results)
    
    for doc in all_documents:
        X.append([norm_bm25.get(doc, 0), norm_dense.get(doc, 0)])
        y.append(1 if doc in relevant_docs[query] else 0)

# Learn weights via logistic regression (or any classifier)
clf = LogisticRegression()
clf.fit(X, y)

# Extract weights
w_bm25, w_dense = clf.coef_[0]
w_bm25 /= (w_bm25 + w_dense)  # Normalize to sum to 1
w_dense = 1 - w_bm25

print(f"Optimal weights: BM25={w_bm25:.2f}, Dense={w_dense:.2f}")

In practice, learned weights often converge to balanced values (0.4–0.6 for each method on general corpora) or skew heavily toward one method if it dominates your query distribution. If you do not have labeled data, empirical tuning on a small sample (e.g., 10–20 hand-judged queries) typically yields good results.

Implementing Hybrid Fusion in Python​

def reciprocal_rank_fusion(bm25_results, dense_results, k=60):
    """
    Fuse two ranked lists using Reciprocal Rank Fusion.
    
    Args:
        bm25_results: List of (doc_id, score) tuples from BM25
        dense_results: List of (doc_id, score) tuples from dense retrieval
        k: RRF constant (default 60)
    
    Returns:
        List of (doc_id, rrf_score) sorted by score descending
    """
    rrf_scores = {}
    
    # Process BM25 results
    for rank, (doc_id, _) in enumerate(bm25_results, 1):
        rrf_scores[doc_id] = rrf_scores.get(doc_id, 0) + 1 / (k + rank)
    
    # Process dense results
    for rank, (doc_id, _) in enumerate(dense_results, 1):
        rrf_scores[doc_id] = rrf_scores.get(doc_id, 0) + 1 / (k + rank)
    
    # Sort by RRF score descending
    fused = sorted(rrf_scores.items(), key=lambda x: x[1], reverse=True)
    return fused

def weighted_normalization_fusion(bm25_results, dense_results, w_bm25=0.5):
    """
    Fuse using weighted normalized scores.
    """
    w_dense = 1 - w_bm25
    
    # Normalize BM25 scores
    bm25_scores = [score for _, score in bm25_results]
    bm25_min, bm25_max = min(bm25_scores), max(bm25_scores)
    norm_bm25 = {doc: (score - bm25_min) / (bm25_max - bm25_min) 
                 for doc, score in bm25_results}
    
    # Normalize dense scores
    dense_scores = [score for _, score in dense_results]
    dense_min, dense_max = min(dense_scores), max(dense_scores)
    norm_dense = {doc: (score - dense_min) / (dense_max - dense_min) 
                  for doc, score in dense_results}
    
    # Combine
    hybrid_scores = {}
    for doc in set(list(norm_bm25.keys()) + list(norm_dense.keys())):
        hybrid_scores[doc] = (w_bm25 * norm_bm25.get(doc, 0) + 
                              w_dense * norm_dense.get(doc, 0))
    
    fused = sorted(hybrid_scores.items(), key=lambda x: x[1], reverse=True)
    return fused

# Example usage
bm25_results = [('doc1', 35.2), ('doc2', 28.1), ('doc3', 22.4)]
dense_results = [('doc1', 0.89), ('doc2', 0.85), ('doc4', 0.81)]

rrf_fused = reciprocal_rank_fusion(bm25_results, dense_results)
print("RRF Fusion:", rrf_fused)

weighted_fused = weighted_normalization_fusion(bm25_results, dense_results, w_bm25=0.5)
print("Weighted Fusion:", weighted_fused)

Fusion Strategies Comparison​

StrategyParameter-FreeInterpretableAccuracyUse Case
Reciprocal Rank FusionYesYes7.5/10Default, no tuning
Weighted NormalizationNoYes8.0/10With labeled eval set
Min-Max + AverageYesYes7.3/10Simple baseline
Learned Neural FusionNoNo8.5/10High-accuracy systems

Key Takeaways​

  • Score normalization is essential for fusing BM25 and dense scores, which operate on different scales.
  • Reciprocal Rank Fusion (RRF) is the most practical method: parameter-free, robust, and requires no training data.
  • Weighted normalization with learned coefficients improves accuracy by 2–5% if you have labeled relevance data but adds tuning complexity.
  • For production RAG systems without labeled data, start with RRF; for systems with evaluation sets, experiment with weighted fusion to find optimal weights.
  • Fusion alone improves over single-method retrieval by 10–20%; adding cross-encoder reranking (Article 6) improves another 5–15%.

Frequently Asked Questions​

What is the best value for the k constant in RRF?​

k=60 is a proven default that works across diverse domains. Theoretical analysis (Cormack et al., 2009) shows k in [40, 100] perform similarly. If your top-50 documents from both methods are very different (high disagreement), try k=100 to reduce rank-order sensitivity. If they are similar, k=60 suffices.

Should I use RRF or learned weights for my RAG system?​

Start with RRF (zero tuning required). If you can label 50–100 query-document pairs as relevant or not, use weighted normalization or logistic regression to optimize weights; expect 2–3% accuracy improvement. For production systems without labels, RRF is sufficient.

Can I fuse more than two retrieval methods?​

Yes. RRF and weighted normalization extend naturally to N methods. Combine BM25, dense retrieval, and lexical-semantic hybrids (e.g., SPLADE) by adding their rank contributions or normalized scores. Each additional method adds marginal accuracy (typically 1–2%) but increases latency linearly.

What if one retrieval method returns very few results (e.g., dense returns 30 results, BM25 returns 100)?​

RRF will downweight the method that returns fewer results because the missing documents get zero contribution. For balanced fusion, pad shorter lists to the same length using zero scores. Alternatively, use a combined pool size (e.g., top-100 from each) and RRF on that unified pool.

How do I handle documents that appear in only one ranking?​

In RRF, a document appearing only in BM25's top-50 gets contribution from BM25 only; dense retrieval contributes zero. This is by design—RRF rewards documents that appear in multiple rankings. In weighted normalization, clip missing scores to 0 or compute missing scores as the average of present scores (conservative estimate).

Further Reading​