MTEB STS (Semantic Textual Similarity)

Correlates embedding cosine similarity with human-rated sentence-pair similarity scores, across 10 datasets spanning up to 18 languages.

Also known as: MTEB Semantic Textual Similarity

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.
Categoryembedding
Subcategorysemantic textual similarity
Page statusactive
MetricSpearman correlation (cosine similarity)
Directionhigher_is_better
Unit%
Dataset size10
Dataset licenceVaries by dataset.
PublisherHugging Face and Cohere (original authors); maintained today by the open embeddings-benchmark community

What it measures

Two sentences are embedded and their cosine similarity is computed, then checked against how well that similarity ranks sentence pairs the same way human annotators' graded similarity scores do.

Task format

Embed sentence pairs; compute cosine similarity; correlate (Spearman) against human similarity ratings.

Models reporting this benchmark

These figures come from the model cards, which carry one collection date per card and no per-score attribution. They are shown as reported, not as verified evidence.
ModelProviderScoreCard as of
jina embeddings v4Jina AI86.22026-04
jina embeddings v4 vllm retrievalJina AI86.22026-04
jina embeddings v5 text nanoJina AI86.22026-04
jina embeddings v5 text smallJina AI86.22026-04
jina embeddings v5 text small retrievalJina AI86.22026-04
jina clip v2Jina AI85.82026-04
jina code embeddings 1.5BJina AI85.82026-04
jina embeddings v3Jina AI85.82026-04
e5 mistral 7B instructintfloat85.52026-04
multilingual e5 large instructintfloat84.82026-04
Gemini Embedding 001Google DeepMind84.52026-04
SFR Embedding 2 RSalesforce84.02026-04
Voyage 3Voyage AI84.02026-04
bge multilingual gemma2BAAI83.52026-04
snowflake arctic embed l v2.0Snowflake83.52026-04
e5 large v2intfloat83.02026-04
bge large en v1.5BAAI82.52026-04
colnomic embed multimodal 7BNomic AI82.52026-04
multilingual e5 largeintfloat82.52026-04
nomic embed multimodal 7BNomic AI82.52026-04
nomic embed text v2 moeNomic AI82.52026-04
nomic embed text v2 moe GGUFNomic AI82.52026-04
bge m3BAAI82.02026-04
e5 largeintfloat82.02026-04
e5 large unsupervisedintfloat82.02026-04
granite embedding english r2IBM82.02026-04
granite embedding small english r2IBM82.02026-04
snowflake arctic embed lSnowflake82.02026-04
bge large enBAAI81.52026-04
snowflake arctic embed m v2.0Snowflake81.52026-04
text-embedding-3-largeOpenAI81.52026-04
Voyage Multilingual 2Voyage AI81.52026-04
Qwen3 Embedding 8BAlibaba / Qwen Team81.12026-04
Qwen3 VL Embedding 8BAlibaba / Qwen Team81.12026-04
e5 base v2intfloat81.02026-04
modernbert embed baseNomic AI81.02026-04
nomic embed text v1.5Nomic AI81.02026-04
nomic embed text v1.5 GGUFNomic AI81.02026-04
Voyage Code 3Voyage AI81.02026-04
Voyage Finance 2Voyage AI81.02026-04
Qwen3 Embedding 4BAlibaba / Qwen Team80.92026-04
Voyage Law 2Voyage AI80.82026-04
bge base en v1.5BAAI80.52026-04
e5 baseintfloat80.52026-04
granite embedding 278M multilingualIBM80.52026-04
snowflake arctic embed m v1.5Snowflake80.52026-04
Voyage 3 LiteVoyage AI80.52026-04
nomic embed text v1Nomic AI80.22026-04
multilingual e5 baseintfloat80.02026-04
snowflake arctic embed mSnowflake80.02026-04
snowflake arctic embed m longSnowflake80.02026-04
NV Embed v2NVIDIA79.62026-04
all mpnet base v2Sentence Transformers79.52026-04
bge base enBAAI79.52026-04
SFR Embedding Code 400M RSalesforce79.52026-04
e5 small v2intfloat79.02026-04
Mistral EmbedMistral AI79.02026-04
all MiniLM L12 v2Sentence Transformers78.82026-04
bge code v1BAAI78.52026-04
e5 smallintfloat78.52026-04
granite embedding 125M englishIBM78.52026-04
all MiniLM L6 v2Sentence Transformers78.22026-04
bge small en v1.5BAAI78.02026-04
CodeRankEmbedNomic AI78.02026-04
multilingual e5 smallintfloat78.02026-04
nomic embed codeNomic AI78.02026-04
snowflake arctic embed sSnowflake78.02026-04
paraphrase mpnet base v2Sentence Transformers77.82026-04
granite embedding 107M multilingualIBM77.52026-04
multi qa mpnet base dot v1Sentence Transformers77.52026-04
Qwen3 Embedding 0.6BAlibaba / Qwen Team77.52026-04
Qwen3 VL Embedding 2BAlibaba / Qwen Team77.52026-04
all roberta large v1Sentence Transformers77.22026-04
multi qa mpnet base cos v1Sentence Transformers77.22026-04
text-embedding-3-smallOpenAI77.22026-04
bge small enBAAI77.02026-04
all distilroberta v1Sentence Transformers76.52026-04
paraphrase multilingual mpnet base v2Sentence Transformers76.52026-04
multi qa MiniLM L6 cos v1Sentence Transformers76.02026-04
snowflake arctic embed xsSnowflake76.02026-04
msmarco bert base dot v5Sentence Transformers75.52026-04
paraphrase MiniLM L12 v2Sentence Transformers75.22026-04
granite embedding 30M englishIBM75.02026-04
granite embedding 30M sparseIBM75.02026-04
paraphrase multilingual MiniLM L12 v2Sentence Transformers75.02026-04
msmarco MiniLM L12 cos v5Sentence Transformers74.52026-04
paraphrase MiniLM L6 v2Sentence Transformers74.52026-04
text-embedding-ada-002OpenAI74.52026-04
msmarco MiniLM L6 v3Sentence Transformers74.02026-04
LaBSESentence Transformers73.02026-04
distiluse base multilingual cased v2Sentence Transformers72.02026-04
paraphrase MiniLM L3 v2Sentence Transformers72.02026-04
distiluse base multilingual cased v1Sentence Transformers70.52026-04

Data

This page as JSON · Edit on GitHub