MMLU: Moral Scenarios

Accuracy on MMLU's moral scenarios questions, one of 57 subject tests of academic and professional knowledge.

Also known as: moral_scenarios

unassessed

What it measures

Short everyday scenarios describing an action, each paired with four judgements of whether that action is morally permissible or wrong under ordinary moral standards; it is MMLU's largest single subject test. Framed as four-option multiple-choice questions and scored zero-shot or few-shot by exact match against the labelled option, as one of the 57 subject subsets that make up the MMLU benchmark.

Task format

Four-option multiple-choice question answering (A-D), one correct answer, zero-shot or few-shot.

Models reporting this benchmark

Data

This page as JSON · Edit on GitHub