{{ message }}
Repository navigation
Run benchmarking with the supported models on BEIR MSMARCO #177
Description
We need to run benchmarking on the BEIR MSMARCO dataset, to have a better understanding of how the models are performing for retrieval tasks.
We can use the test split available on Hugging Face hub:
Proposed metrics:
- NDCG@10
- Precision@10
- Recall@100
Considering non-judged documents as non-relevant.
Activity
HAKSOAT commentedon Jan 17, 2025
The
InformationRetrievalEvaluatorfrom SentenceTransformers can be helpful for this.