MMLU: High School Microeconomics

Accuracy on MMLU's high school microeconomics questions, one of 57 subject tests of academic and professional knowledge.

Also known as: high_school_microeconomics

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.
Categoryknowledge
SubcategorySocial Sciences
Page statusactive
Metricaccuracy
Directionhigher_is_better
Unit%
Dataset size238
Dataset licenceMIT
PublisherUC Berkeley

What it measures

Introductory microeconomics at the level of a US high-school or AP course: supply and demand, elasticity, market structures, opportunity cost, and firm and consumer behaviour. Framed as four-option multiple-choice questions and scored zero-shot or few-shot by exact match against the labelled option, as one of the 57 subject subsets that make up the MMLU benchmark.

Task format

Four-option multiple-choice question answering (A-D), one correct answer, zero-shot or few-shot.

Models reporting this benchmark

These figures come from the model cards, which carry one collection date per card and no per-score attribution. They are shown as reported, not as verified evidence.
ModelProviderScoreCard as of
Meta Llama 3 70BMeta88.72024-07
Meta Llama 3 70B InstructMeta88.72026-04
Meta Llama 3 70B InstructNous Research88.72026-04
Yi 1.5 34B01.AI88.72026-04
Yi 1.5 34B 32K01.AI88.72024-07
Mixtral 8x22B Instruct v0.1Mistral AI87.82026-04
Yi 1.5 34B Chat01.AI87.82026-04
Yi 1.5 34B Chat 16K01.AI87.82024-07
Yi 1.5 9B Chat01.AI87.02024-07
Yi 1.5 9B01.AI86.12024-07
Nous Hermes 2 Yi 34BNous Research85.32024-07
Yi 1.5 9B Chat 16K01.AI84.52024-07
Yi 34B 200K01.AI84.02024-07
Phi 3 mini 4K instructMicrosoft83.62024-07
Yi 34B Chat01.AI83.62024-07
Phi 3 mini 128K instructMicrosoft82.82024-07
Yi 1.5 6B Chat01.AI82.82024-07
Yi 1.5 9B 32K01.AI82.82024-07
Yi 9B01.AI82.42024-07
Yi 1.5 6B01.AI81.92024-07
Mixtral 8x7B Instruct v0.1Mistral AI80.32026-04
Nous Hermes 2 Mixtral 8x7B DPONous Research79.42024-07
Mixtral 8x7B v0.1Mistral AI78.62026-04
Meta Llama 3 8B InstructMeta76.52024-07
Meta Llama 3 8B InstructNous Research76.52024-07
Hermes 2 Theta Llama 3 8BNous Research75.22024-07
Yi 6B01.AI73.92024-07
Yi 6B Chat01.AI73.92024-07
Meta Llama 3 8BMeta73.52024-07
Meta Llama 3 8BNous Research73.52024-07
Hermes 2 Pro Llama 3 8BNous Research72.72024-07
Nous Hermes 2 SOLAR 10.7BNous Research69.32024-07
gemma 7B itGoogle DeepMind66.82024-07
Mistral 7B Instruct v0.2Mistral AI65.52024-07
Mistral 7B v0.3Mistral AI63.92024-07
mistral 7B v0.3 bnb 4bitUnsloth63.92024-07
phi 2Microsoft62.22024-07
Qwen2 1.5B InstructAlibaba / Qwen Team62.22024-07
falcon 40BTII55.52024-07
deepseek llm 7B baseDeepSeek47.52024-07
deepseek llm 7B chatDeepSeek47.52024-07
Qwen2 0.5B InstructAlibaba / Qwen Team47.52024-07
chatglm2 6BZhipu AI43.32024-07
deepseek coder 6.7B instructDeepSeek41.22024-07
gemma 2BGoogle DeepMind38.72024-07
deepseek coder 6.7B baseDeepSeek36.12024-07
gemma 2B itGoogle DeepMind34.02024-07
deepseek coder 1.3B baseDeepSeek26.12024-07
deepseek coder 1.3B instructDeepSeek26.12024-07
OLMo 1B hfAllen AI21.42024-07

Data

This page as JSON · Edit on GitHub