Parallel extractive question answering in seven languages, testing whether a model can recover an answer span from Wikipedia even when the question is in a different language.
unassessed
| Category | knowledge |
|---|---|
| Subcategory | multilingual extractive question answering |
| Page status | active |
| Metric | token F1 and exact match, using the official MLQA evaluation script |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 12738 |
| Dataset licence | CC-BY-SA-3.0 |
| Publisher | Facebook AI Research |
MLQA (MultiLingual Question Answering) is an evaluation-only extractive QA set. The model receives a Wikipedia paragraph and a question, and must return the answer as a span in the paragraph's language. Seven languages are covered: English, Arabic, German, Spanish, Hindi, Vietnamese and Simplified Chinese. Instances are multi-way parallel, on average across four languages, so the same question can be asked against a context in another language. That is the generalised transfer setup. It is a reading-span task in SQuAD format, not a translation benchmark, and not the medical acronyms MEDIQA or MedQA.
Extractive span selection in SQuAD JSON. lm-evaluation-harness instead generates the answer string from a "Context / Question / Answer" prompt and scores it with the official MLQA normalisation (language-specific article stripping and Chinese mixed segmentation).
No model card in ModelSpec reports this benchmark yet.