MMLU: Logical Fallacies

Accuracy on MMLU's logical fallacies questions, one of 57 subject tests of academic and professional knowledge.

Also known as: logical_fallacies

unassessed

What it measures

Identification of informal fallacies in short arguments -- ad hominem, straw man, false dilemma, and the like -- at the level of an introductory critical-thinking or informal-logic course. Framed as four-option multiple-choice questions and scored zero-shot or few-shot by exact match against the labelled option, as one of the 57 subject subsets that make up the MMLU benchmark.

Task format

Four-option multiple-choice question answering (A-D), one correct answer, zero-shot or few-shot.

Models reporting this benchmark

Data

This page as JSON · Edit on GitHub