The easier, larger half of the AI2 Reasoning Challenge; questions that 2018-era retrieval or word-overlap baselines could already answer, now scored near ceiling by current models.
unassessed
| Category | reasoning |
|---|---|
| Subcategory | grade-school science multiple-choice QA (Easy split) |
| Page status | superseded |
| Metric | accuracy (often reported as acc_norm, length-normalised) |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 5197 |
| Dataset licence | CC-BY-SA-4.0 |
| Publisher | Allen Institute for AI (AI2) |
ARC-Easy is the larger of the two AI2 Reasoning Challenge splits: 5,197 grade-school-level natural science exam questions that landed here specifically because at least one of two 2018-era baseline solvers (an information-retrieval solver or a word-co-occurrence/PMI solver) answered them correctly. Every question that both baselines failed went to the harder ARC-Challenge split instead. Format, source and authorship are identical to Challenge; only the difficulty filter differs.
Multiple-choice science question, typically 4 answer options, single correct answer.
No model card in ModelSpec reports this benchmark yet.