MMLU: High School Physics

Accuracy on MMLU's high school physics questions, one of 57 subject tests of academic and professional knowledge.

Also known as: high_school_physics

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.
Categoryknowledge
SubcategorySTEM
Page statusactive
Metricaccuracy
Directionhigher_is_better
Unit%
Dataset size151
Dataset licenceMIT
PublisherUC Berkeley

What it measures

Introductory, largely algebra-based physics: mechanics, waves and sound, electricity and magnetism, and basic thermodynamics, at the level of a US high-school course. Framed as four-option multiple-choice questions and scored zero-shot or few-shot by exact match against the labelled option, as one of the 57 subject subsets that make up the MMLU benchmark.

Task format

Four-option multiple-choice question answering (A-D), one correct answer, zero-shot or few-shot.

Models reporting this benchmark

These figures come from the model cards, which carry one collection date per card and no per-score attribution. They are shown as reported, not as verified evidence.
ModelProviderScoreCard as of
Meta Llama 3 70BMeta62.32024-07
Meta Llama 3 70B InstructMeta62.32026-04
Meta Llama 3 70B InstructNous Research62.32026-04
Yi 1.5 34B Chat01.AI60.92026-04
Yi 1.5 34B Chat 16K01.AI60.92024-07
Yi 1.5 34B01.AI55.62026-04
Yi 1.5 34B 32K01.AI54.32024-07
Yi 1.5 9B Chat 16K01.AI51.02024-07
Mixtral 8x22B Instruct v0.1Mistral AI50.32026-04
Nous Hermes 2 Yi 34BNous Research50.32024-07
Yi 34B Chat01.AI50.32024-07
Mixtral 8x7B v0.1Mistral AI49.02026-04
Yi 1.5 9B01.AI47.72024-07
Mixtral 8x7B Instruct v0.1Mistral AI47.02026-04
Yi 1.5 9B 32K01.AI46.42024-07
Yi 34B 200K01.AI46.42024-07
Nous Hermes 2 Mixtral 8x7B DPONous Research45.72024-07
Yi 1.5 9B Chat01.AI45.72024-07
Hermes 2 Theta Llama 3 8BNous Research45.02024-07
Meta Llama 3 8BMeta44.42024-07
Meta Llama 3 8BNous Research44.42024-07
Meta Llama 3 8B InstructMeta44.42024-07
Meta Llama 3 8B InstructNous Research44.42024-07
Phi 3 mini 128K instructMicrosoft44.42024-07
Phi 3 mini 4K instructMicrosoft44.42024-07
Yi 1.5 6B01.AI44.42024-07
gemma 7B itGoogle DeepMind41.72024-07
Yi 9B01.AI41.12024-07
Hermes 2 Pro Llama 3 8BNous Research39.12024-07
Yi 1.5 6B Chat01.AI39.12024-07
Yi 6B01.AI39.12024-07
Yi 6B Chat01.AI39.12024-07
phi 2Microsoft38.42024-07
Nous Hermes 2 SOLAR 10.7BNous Research37.72024-07
Mistral 7B Instruct v0.2Mistral AI35.82024-07
Mistral 7B v0.3Mistral AI35.82024-07
mistral 7B v0.3 bnb 4bitUnsloth35.82024-07
Qwen2 1.5B InstructAlibaba / Qwen Team35.82024-07
deepseek coder 6.7B instructDeepSeek33.12024-07
falcon 40BTII31.82024-07
OLMo 1B hfAllen AI31.12024-07
deepseek coder 1.3B baseDeepSeek30.52024-07
deepseek coder 1.3B instructDeepSeek30.52024-07
chatglm2 6BZhipu AI29.12024-07
deepseek llm 7B baseDeepSeek29.12024-07
deepseek llm 7B chatDeepSeek29.12024-07
deepseek coder 6.7B baseDeepSeek27.82024-07
Qwen2 0.5B InstructAlibaba / Qwen Team27.82024-07
gemma 2B itGoogle DeepMind25.82024-07
gemma 2BGoogle DeepMind25.22024-07

Data

This page as JSON · Edit on GitHub