Evaluating Retrieval Quality: MRR, NDCG, and Hit Rate in Hybrid Search Systems

In Retrieval-Augmented Generation (RAG) and hybrid search knowledge platforms, optimizing search relevance requires moving beyond subjective "vibes" to rigorous, quantifiable Information Retrieval (IR) evaluation metrics.

This guide details the mathematical definitions of Mean Reciprocal Rank (MRR), Normalized Discounted Cumulative Gain (NDCG@K), Hit Rate@K, and automated regression testing harnesses.

---

1. Quick-Reference: Key IR Evaluation Metrics

+-----------------------------------------------------------------------------------------+
|                               INFORMATION RETRIEVAL EVALUATION METRICS                  |
+-----------------------------------------------------------------------------------------+
| Metric                 | Mathematical Focus                | Target Benchmark Standard  |
+------------------------+-----------------------------------+----------------------------+
| Mean Reciprocal Rank   | Inverse rank of 1st relevant item | MRR > 0.85 (High Precision)|
| (MRR)                  | $	ext{MRR} = rac{1}{|Q|} \sum rac{1}{	ext{rank}_i}$ | |
| Hit Rate @ K           | % of queries finding target in top K| Hit@5 > 95%               |
| NDCG @ K               | Graded relevance with log discount| NDCG@10 > 0.80             |
+-----------------------------------------------------------------------------------------+

---

2. Automated Regression Test Harness

The Wikantik retrieval test harness executes a suite of 200+ canonical developer queries daily, comparing Lucene BM25, vector embeddings, and Reciprocal Rank Fusion (RRF) outputs against ground-truth document labels to detect ranking regressions before deployment.