35,727 multiple-choice questions from real Basque public-service admission exams, parallel in Basque (16,774) and Spanish (18,953), from the Latxa evaluation suite.
unassessed
| Category | knowledge |
|---|---|
| Subcategory | Basque and Spanish public-service exam multiple-choice QA (civil-service admission tests) |
| Page status | active |
| Metric | accuracy (acc and acc_norm, length-normalised) |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 35727 |
| Dataset licence | CC BY-SA 4.0, per the Hugging Face dataset card's licence tag |
| Publisher | HiTZ Center - Ixa, University of the Basque Country (UPV/EHU) |
EusExams tests whether a model can answer the kind of multiple-choice question used in real admission exams for Basque public-sector jobs -- civil-service tests set by the Basque Government, the public health system Osakidetza, the City Councils of Bilbao and Gasteiz (Vitoria), and the University of the Basque Country (UPV/EHU), across roles from administrative assistant to technician and nurse. Each question has 2 to 4 answer options (3.90 on average) and one correct answer. Because the questions were written for real civil-service candidates rather than for language-model evaluation, they draw on general knowledge, procedural and legal/administrative material rather than any single narrow skill.
Multiple-choice question with up to four labelled options (A-D), single correct answer, offered in parallel Basque-language and Spanish-language versions of largely the same underlying exams; scored by loglikelihood ranking over the (up to four) candidate options.
No model card in ModelSpec reports this benchmark yet.