Multiple-choice Arabic high-school exam questions across five subjects, run under two different harness names (arabic_exams, aexams) over the same 562-item set.
unassessed
| Category | knowledge |
|---|---|
| Subcategory | Arabic high-school exam multiple-choice QA (5 subjects) |
| Page status | active |
| Metric | accuracy (lm-evaluation-harness also reports acc_norm; HELM's main metric is exact_match) |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 562 |
Arabic EXAMS gives a model a high-school-level exam question written in Arabic, with four labelled answer options, and asks it to pick the correct one. The questions are drawn from real school examinations rather than written for the benchmark, and cover five subjects -- Islamic Studies, Biology, Physics, Science, and Social (Studies) -- so a score on this benchmark reflects a mix of Arabic reading comprehension and subject-matter recall rather than any single skill. It is the Arabic-language slice of EXAMS, a multilingual high-school-exam question answering dataset (Hardalov et al., 2020), later re-packaged for LLM evaluation by the AceGPT project.
Multiple-choice exam question with four labelled options (A-D), single correct answer, in Arabic; scored either by log-likelihood ranking over the options or by exact match on a generated letter, depending on harness.
No model card in ModelSpec reports this benchmark yet.