FLORES-200

A human-translated, sentence-aligned evaluation set spanning 200 languages, used to score machine-translation quality between any language pair.

Also known as: FLORES, Flores-200, FLORES+, No Language Left Behind evaluation benchmark

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	translation
Subcategory	machine translation
Page status	active
Metric	BLEU
Direction	higher_is_better
Unit	%
Dataset size	3001
Dataset licence	CC BY-SA 4.0
Publisher	Meta AI (FAIR), NLLB Team; now maintained by the community-run Open Language Data Initiative (OLDI)

What it measures

FLORES-200 measures machine translation quality: for a chosen source and target language, a system translates the same set of held-out sentences that professional translators produced in every one of the benchmark's 200 languages, and the output is compared to the human reference. Because every language shares the same underlying sentence set, scores are directly comparable across language pairs, including many low-resource pairs with no other public evaluation data. This repository's language-pair subsets (flores_en_de, flores_en_es, flores_en_ja, flores_en_zh) each cover one English-source direction.

Task format

Translate each devtest sentence from a source language into a target language; compare machine output to the human reference with an automatic metric.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub