Metrics API

evret.metrics.base

Base interface for retrieval evaluation metrics.

Metric

Bases: ABC

Base class for metrics evaluated at a top-k cutoff.

For query i with retrieved document IDs R_i and relevant IDs G_i, each metric computes a per-query score at k and then averages:

score = (1 / |Q|) * sum(metric_i(R_i[:k], G_i))

name property

Metric display name including cutoff.

score(retrieved_by_query, relevant_by_query)

Score a batch of queries by averaging per-query metric values.

score_query(retrieved_doc_ids, relevant_doc_ids) abstractmethod

Score a single query.

top_k(retrieved_doc_ids)

Return the retrieval list trimmed to metric cutoff.

evret.metrics.hit_rate

Hit Rate metric implementation.

HitRate

Bases: Metric

Binary top-k relevance presence metric.

Formula: HitRate@k = (1 / |Q|) * sum(1[relevant_i ∩ retrieved_i[:k] != ∅])

evret.metrics.recall

Recall@K metric implementation.

Recall

Bases: Metric

Coverage metric over relevant documents.

Formula: Recall@k = |relevant ∩ retrieved[:k]| / |relevant|

evret.metrics.precision

Precision@K metric implementation.

Precision

Bases: Metric

Purity metric over retrieved top-k documents.

Formula: Precision@k = |relevant ∩ retrieved[:k]| / k

evret.metrics.mrr

MRR@K metric implementation.

MRR

Bases: Metric

Mean Reciprocal Rank query metric at top-k.

Formula: RR@k = 1 / rank_first_relevant if a hit exists in top-k, else 0.

evret.metrics.ndcg

nDCG@K metric implementation.

NDCG

Bases: Metric

Normalized Discounted Cumulative Gain at top-k.

evret.metrics.average_precision

Average Precision@K metric implementation.

AveragePrecision

Bases: Metric

Average Precision at top-k cutoff.