5,957 four-way elementary-science questions built around a small fact book; human accuracy is near 92%, and no current model card or live leaderboard still reports it.
unverified
Recorded reasons:
| Category | reasoning |
|---|---|
| Subcategory | open-book elementary-science multiple-choice QA |
| Page status | saturated |
| Metric | accuracy (acc and length-normalised acc_norm) |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 5957 |
| Dataset licence | Marked "unknown" on the Hugging Face dataset card. The reference code repository (allenai/OpenBookQA on GitHub) is Apache-2.0 licensed, but that licence covers the baseline modelling code, not a separately stated licence for the question-and-fact text itself. |
| Publisher | Allen Institute for Artificial Intelligence (AI2) |
OpenBookQA asks a model to answer an elementary-school-level science question by combining one "open book" fact with broad common knowledge, the way an open-book exam works: knowing where to look up the fact is not enough, because the missing piece is background knowledge no book states outright. The open book itself is a set of roughly 1,326-1,329 core science facts (sources read for this page give slightly different counts; see Dataset and licence) covering topics such as conductivity, states of matter and food chains. Around 6,000 four-way multiple-choice questions were then written against those facts so that each one also requires an additional, unstated piece of common knowledge to connect the fact to the question -- for example, the fact "metal is a thermal conductor" only answers a question about which object conducts the most heat if the model also knows a steel spoon is made of metal. That two-hop structure sets it apart from earlier reading-comprehension datasets, which typically expect an answer extractable from one supplied passage.
Four-way multiple-choice question answering, typically zero- or few-shot, over a fixed set of roughly 6,000 English-language elementary-science questions; no passage or fact is supplied at inference time, so the model must supply both the fact and the connecting common-knowledge step itself.
No model card in ModelSpec reports this benchmark yet.