1,715 Basque trivia questions (2-4 options, 3.84 on average) from online sources, 56.3% elementary, with a large Basque Country, language and culture share.
unassessed
| Category | knowledge |
|---|---|
| Subcategory | Basque trivia multiple-choice across five knowledge areas, partly Basque-local |
| Page status | active |
| Metric | accuracy |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 1715 |
| Dataset licence | Not stated on the Hugging Face card (no licence tag). The Latxa paper says the evaluation datasets are "publicly available under open licenses" without naming one. This repository's BasqueBench page independently recorded that EusTrivia's card has no licence tag. |
| Publisher | HiTZ Center - Ixa, University of the Basque Country (UPV/EHU) |
EusTrivia tests general and Basque-local factual knowledge in Basque. Questions come from several online trivia sources. 56.3% are labelled elementary (grades 3-6); the rest are described as challenging. A substantial share focuses on the Basque Country, its language and culture. Five areas are used: Humanities and Natural Sciences (27.8%), Leisure and Art (24.5%), Music (16.0%), Language and Literature (17.1%), and Mathematics and ICT (14.5%). Each item has two, three or four options (3.84 on average) and one correct answer.
Variable-arity multiple-choice in Basque. The harness prompt is `Galdera: {question}` plus as many of A-D as exist, then `Erantzuna:`. The Latxa paper evaluates it 5-shot by log-probability over those letters.
No model card in ModelSpec reports this benchmark yet.