RACE is multiple-choice reading comprehension from English exams written for Chinese middle- and high-school students, split into easier RACE-M and harder RACE-H passages, four options per question.
unassessed
| Category | reasoning |
|---|---|
| Subcategory | multiple-choice reading comprehension: English-exam questions written for Chinese middle- and high-school students |
| Page status | active |
| Metric | Accuracy (percentage of questions answered correctly) |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 97687 |
| Dataset licence | Non-commercial research use only (custom licence; Hugging Face tags it 'other' rather than a standard SPDX identifier) |
| Publisher | Carnegie Mellon University |
RACE gives a model a passage taken from an English-language exam administered to Chinese students aged roughly 12 to 18, together with a question about it and four answer options, exactly as the question appeared on the real exam. Questions come in two forms: ordinary interrogative questions, and cloze-style questions with an underscore standing in for a missing phrase that the correct option must complete. The passages were written by English teachers specifically to test the students' reading comprehension and reasoning, not sampled from generic web text, so RACE leans more on inference, vocabulary and paraphrase than on locating a matching sentence the way span-extraction sets like SQuAD (`squad`) do. The dataset is split by school level into RACE-M (middle school, easier) and RACE-H (high school, harder); a combined "all" configuration pools both.
Four-way multiple choice: given a passage and a question (or cloze sentence), select the correct one of four lettered options (A-D). Harness implementations vary in whether they present this as a loglikelihood ranking over the four options or a generation task asking the model to output a letter.
No model card in ModelSpec reports this benchmark yet.