MMLU: Chemistry (subcategory)

The chemistry subcategory of MMLU: a rollup of the College Chemistry and High School Chemistry subjects, used by publishers that report MMLU at a coarser grain than all 57 subjects.

unassessed

What it measures

This id does not correspond to a single dataset config in the Hugging Face mirror of MMLU. It corresponds to the "chemistry" subcategory the benchmark's authors define in the repository's categories.py, which pools the College Chemistry and High School Chemistry subjects -- general and organic chemistry at an undergraduate level, and standard high-school chemistry respectively. Some publishers report MMLU broken down by this kind of subcategory rather than by all 57 individual subjects; this id captures scores reported at that grain.

Task format

Four-option multiple-choice questions pooled from two underlying MMLU subjects (College Chemistry, High School Chemistry), graded on the single correct labelled option. How a given publisher averages the two subjects into one number -- an unweighted mean of per-subject accuracy, or a single accuracy over the pooled question set -- is not documented and not established here.

Models reporting this benchmark

Data

This page as JSON · Edit on GitHub