OpenBookQA

OpenBookQA tests multi-step science question answering with a small open book of facts and four answer choices.

Also known as: OBQA

alias

This is an alternate identifier for another benchmark, not a separate benchmark.

Recorded reasons:

independently reviewed alias; retain canonical benchmark and protocol distinctions

Canonical page: openbookqa

Category	knowledge
Subcategory	open-book question answering
Page status	active
Metric	accuracy
Direction	higher_is_better
Unit	percent
Dataset size	500
Publisher	Allen Institute for AI

What it measures

OpenBookQA asks grade-school science questions that require combining a supplied salient fact with common knowledge and language understanding. It is designed as an open-book exam style evaluation.

Task format

Four-choice multiple-choice question, optionally accompanied by a fact; output A, B, C, or D.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub