A multi-task suite that scores text embedding models on retrieval, classification, clustering, reranking, similarity, summarization and pair classification.
unassessed
| Category | embedding |
|---|---|
| Subcategory | text embedding |
| Page status | active |
| Direction | higher_is_better |
| Dataset licence | Varies by dataset (each task keeps its source dataset's own licence); the MTEB benchmark code itself is Apache-2.0. |
| Publisher | Hugging Face and Cohere (original authors); maintained today by the open embeddings-benchmark community |
MTEB measures how useful a text embedding model's output vectors are for downstream work, not how well the model writes text. Each task type applies a fixed, non-learned scoring routine to the embeddings a model produces: nearest-neighbour classification, clustering, cosine-similarity ranking for retrieval and reranking, or correlation with human similarity judgements. The task types in scope for this repository are retrieval, classification, clustering, reranking, semantic textual similarity, summarization and pair classification; bitext mining and the newer image, audio and video MTEB variants sit outside this family page's scope. Coverage ranges from English-only task sets to multilingual ones spanning over 100 languages.
Varies by task type: query-document ranking for retrieval and reranking, single-text classification into a fixed label set, unsupervised clustering of a text collection, sentence-pair scoring for similarity or entailment, and summary scoring by embedding distance to reference summaries. Every task consumes only the model's embedding vectors.
No model card in ModelSpec reports this benchmark yet.