A 518-item Persian span-extraction BIG-bench Lite task taken from the ParsiNLU reading-comprehension eval split.
unassessed
| Category | knowledge |
|---|---|
| Subcategory | Persian extractive reading comprehension (BIG-bench Lite) |
| Page status | unknown |
| Metric | exact_str_match |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 518 |
| Dataset licence | Apache-2.0 |
| Publisher | BIG-bench collaboration (task authors); ParsiNLU authors |
parsinlu_reading_comprehension gives a Persian passage and a Persian question and asks for the shortest coherent span that answers it. Gold may be one or more strings. Authors Mozhdeh Gheini, Siamak Shakeri, Daniel Khashabi, Sarik Ghazarian, Sepideh Sadeghi, and Yadollah Yaghoobzadeh took the ParsiNLU eval split, dropped 52 sensitive items from 570, and shipped 518. Questions began as Google-autocomplete seeds, then native annotators marked spans in retrieved pages; eval items had two annotators. This is passage QA, not the open-domain quiz on [parsinlu_qa](parsinlu_qa.md).
Free-text span. preferred_score exact_str_match. Keywords: reading comprehension, contextual question-answering, low-resource language. Canary GUID embedded. Dummy-model header: 0 multiple-choice and 518 free-text queries. Listed among the 24 BIG-bench Lite JSON tasks.
No model card in ModelSpec reports this benchmark yet.