MMLU: High School Government and Politics

MMLU subject subset: US government structure, civics and political institutions.

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.
Categoryknowledge
Subcategorypolitics
Page statusactive
Metricaccuracy
Directionhigher_is_better
Unit%
Dataset size193
Dataset licenceMIT
PublisherUC Berkeley (original); Center for AI Safety (current host)

What it measures

US government structure, civics and political institutions. Questions are four-option multiple-choice, drawn from the MMLU test set's "politics" subcategory within the benchmark's "social sciences" top-level group, and are graded on the single correct labelled option.

Task format

Four-option multiple-choice questions, graded on the single correct labelled option; commonly evaluated 5-shot, consistent with the rest of MMLU.

Models reporting this benchmark

These figures come from the model cards, which carry one collection date per card and no per-score attribution. They are shown as reported, not as verified evidence.
ModelProviderScoreCard as of
Meta Llama 3 70BMeta98.42024-07
Meta Llama 3 70B InstructMeta98.42026-04
Meta Llama 3 70B InstructNous Research98.42026-04
Nous Hermes 2 Yi 34BNous Research97.92024-07
Yi 34B 200K01.AI97.42024-07
Mixtral 8x22B Instruct v0.1Mistral AI96.92026-04
Mixtral 8x7B Instruct v0.1Mistral AI95.92026-04
Yi 1.5 34B Chat01.AI95.92026-04
Yi 1.5 34B Chat 16K01.AI95.92024-07
Yi 34B Chat01.AI95.32024-07
Yi 1.5 34B01.AI94.82026-04
Yi 1.5 34B 32K01.AI94.82024-07
Mixtral 8x7B v0.1Mistral AI93.82026-04
Nous Hermes 2 Mixtral 8x7B DPONous Research93.82024-07
Yi 9B01.AI93.82024-07
Meta Llama 3 8B InstructMeta91.22024-07
Meta Llama 3 8B InstructNous Research91.22024-07
Yi 1.5 9B 32K01.AI91.22024-07
Phi 3 mini 4K instructMicrosoft90.72024-07
Yi 1.5 9B01.AI90.22024-07
Yi 1.5 9B Chat01.AI90.22024-07
Nous Hermes 2 SOLAR 10.7BNous Research89.62024-07
Yi 6B01.AI89.62024-07
Yi 6B Chat01.AI89.62024-07
Meta Llama 3 8BMeta89.12024-07
Meta Llama 3 8BNous Research89.12024-07
Yi 1.5 9B Chat 16K01.AI89.12024-07
gemma 7B itGoogle DeepMind88.62024-07
Hermes 2 Theta Llama 3 8BNous Research88.62024-07
Phi 3 mini 128K instructMicrosoft88.62024-07
Hermes 2 Pro Llama 3 8BNous Research88.12024-07
Mistral 7B v0.3Mistral AI87.62024-07
mistral 7B v0.3 bnb 4bitUnsloth87.62024-07
Yi 1.5 6B01.AI87.62024-07
Mistral 7B Instruct v0.2Mistral AI85.52024-07
Yi 1.5 6B Chat01.AI84.52024-07
phi 2Microsoft80.82024-07
Qwen2 1.5B InstructAlibaba / Qwen Team78.82024-07
falcon 40BTII77.22024-07
deepseek llm 7B baseDeepSeek71.52024-07
deepseek llm 7B chatDeepSeek71.52024-07
gemma 2BGoogle DeepMind59.12024-07
chatglm2 6BZhipu AI57.02024-07
Qwen2 0.5B InstructAlibaba / Qwen Team56.02024-07
gemma 2B itGoogle DeepMind47.72024-07
deepseek coder 6.7B instructDeepSeek43.02024-07
deepseek coder 6.7B baseDeepSeek42.52024-07
deepseek coder 1.3B baseDeepSeek32.62024-07
deepseek coder 1.3B instructDeepSeek32.62024-07
OLMo 1B hfAllen AI25.92024-07

Data

This page as JSON · Edit on GitHub