352 Basque reading-comprehension items from EGA irakurmena papers (1998-2008), with long passages that the Latxa paper used as a 1-shot long-context probe.
unassessed
| Category | knowledge |
|---|---|
| Subcategory | Basque EGA irakurmena reading comprehension with long passages (1998-2008) |
| Page status | active |
| Metric | accuracy |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 352 |
| Dataset licence | Not stated on the Hugging Face card (no licence tag). The Latxa paper says the evaluation datasets are "publicly available under open licenses" without naming one. The Latxa GitHub MIT licence is not confirmed as the dataset licence. |
| Publisher | HiTZ Center - Ixa, University of the Basque Country (UPV/EHU) |
EusReading tests whether a model can read a long Basque passage and answer a multiple-choice question about it. The items are the irakurmena (reading) exercises from the same EGA C1 exam papers, 1998 to 2008, that supplied EusProficiency. The Latxa paper says each original test generally had about 10 questions, and that the passages are longer and harder than Belebele's Basque split, so they treat the set as a long-context reading probe rather than a short-span QA quiz.
Passage plus question plus two to four options in Basque. lm-evaluation-harness prefixes the passage with `Pasartea:`, then `Galdera:` and labelled options, ending with `Erantzuna:`. The paper evaluates it 1-shot because more exemplars would not fit most models' context windows, and scores by log-probability over the candidate letters.
No model card in ModelSpec reports this benchmark yet.