5,169 four-option Basque questions from the EGA C1 qualifying paper (atarikoa, 1998-2008), one of four Latxa evaluation-suite tasks.
unassessed
| Category | knowledge |
|---|---|
| Subcategory | Basque C1 EGA language-proficiency exam multiple-choice (atarikoa, 1998-2008) |
| Page status | active |
| Metric | accuracy |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 5169 |
| Dataset licence | Not stated on the Hugging Face card (no licence tag). The Latxa paper says the evaluation datasets are "publicly available under open licenses" without naming one. The Latxa GitHub LICENSE is MIT and covers the project repository; it is not confirmed as the dataset licence. |
| Publisher | HiTZ Center - Ixa, University of the Basque Country (UPV/EHU) |
EusProficiency tests whether a model can pass the kind of multiple-choice item used on the first qualifying paper (atarikoa) of EGA, the official C1 certificate of proficiency in Basque. Items cover reading, grammar, vocabulary, spelling and writing skill as those exams defined them, not a single academic subject. Each question has four labelled options and one correct answer. The Latxa authors present it as a language-proficiency probe that cannot be obtained by translating an English exam, because the certificate is Basque-specific.
Four-option multiple-choice in Basque. lm-evaluation-harness formats each item as `Galdera: {question}` plus A-D options and `Erantzuna:`, then ranks option letters by log-likelihood. The Latxa paper evaluates it 5-shot with the same log-probability rule.
No model card in ModelSpec reports this benchmark yet.