MMLU: Anatomy

MMLU subject subset: Human anatomical structures, organ systems and anatomical terminology, at a premedical or undergraduate level.

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.
Categoryknowledge
Subcategoryhealth
Page statusactive
Metricaccuracy
Directionhigher_is_better
Unit%
Dataset size135
Dataset licenceMIT
PublisherUC Berkeley (original); Center for AI Safety (current host)

What it measures

Human anatomical structures, organ systems and anatomical terminology, at a premedical or undergraduate level. Questions are four-option multiple-choice, drawn from the MMLU test set's "health" subcategory within the benchmark's "other" top-level group, and are graded on the single correct labelled option.

Task format

Four-option multiple-choice questions, graded on the single correct labelled option; commonly evaluated 5-shot, consistent with the rest of MMLU.

Models reporting this benchmark

These figures come from the model cards, which carry one collection date per card and no per-score attribution. They are shown as reported, not as verified evidence.
ModelProviderScoreCard as of
Yi 1.5 34B Chat01.AI80.02026-04
Yi 1.5 34B Chat 16K01.AI80.02024-07
Meta Llama 3 70BMeta77.02024-07
Meta Llama 3 70B InstructMeta77.02026-04
Meta Llama 3 70B InstructNous Research77.02026-04
Yi 1.5 34B01.AI75.62026-04
Yi 1.5 34B 32K01.AI75.62024-07
Mixtral 8x22B Instruct v0.1Mistral AI74.82026-04
Yi 34B 200K01.AI73.32024-07
Mixtral 8x7B v0.1Mistral AI71.92026-04
Nous Hermes 2 Yi 34BNous Research71.92024-07
Yi 34B Chat01.AI71.12024-07
Meta Llama 3 8BMeta69.62024-07
Meta Llama 3 8BNous Research69.62024-07
Nous Hermes 2 Mixtral 8x7B DPONous Research68.92024-07
Yi 1.5 9B Chat 16K01.AI68.12024-07
Mixtral 8x7B Instruct v0.1Mistral AI66.72026-04
Phi 3 mini 4K instructMicrosoft66.72024-07
Yi 1.5 9B01.AI66.72024-07
Yi 1.5 9B 32K01.AI65.92024-07
Meta Llama 3 8B InstructMeta65.22024-07
Meta Llama 3 8B InstructNous Research65.22024-07
Phi 3 mini 128K instructMicrosoft65.22024-07
Hermes 2 Theta Llama 3 8BNous Research63.72024-07
Yi 1.5 9B Chat01.AI63.72024-07
Hermes 2 Pro Llama 3 8BNous Research63.02024-07
Yi 6B01.AI61.52024-07
Yi 6B Chat01.AI61.52024-07
Mistral 7B v0.3Mistral AI60.72024-07
mistral 7B v0.3 bnb 4bitUnsloth60.72024-07
gemma 7B itGoogle DeepMind57.82024-07
Yi 9B01.AI57.82024-07
Mistral 7B Instruct v0.2Mistral AI57.02024-07
Nous Hermes 2 SOLAR 10.7BNous Research56.32024-07
Yi 1.5 6B01.AI55.62024-07
falcon 40BTII53.32024-07
Yi 1.5 6B Chat01.AI51.92024-07
gemma 2BGoogle DeepMind48.92024-07
Qwen2 1.5B InstructAlibaba / Qwen Team48.92024-07
deepseek llm 7B baseDeepSeek45.92024-07
deepseek llm 7B chatDeepSeek45.92024-07
phi 2Microsoft44.42024-07
Qwen2 0.5B InstructAlibaba / Qwen Team43.02024-07
chatglm2 6BZhipu AI41.52024-07
deepseek coder 6.7B baseDeepSeek40.02024-07
gemma 2B itGoogle DeepMind38.52024-07
deepseek coder 6.7B instructDeepSeek34.82024-07
OLMo 1B hfAllen AI32.62024-07
deepseek coder 1.3B baseDeepSeek29.62024-07
deepseek coder 1.3B instructDeepSeek29.62024-07

Data

This page as JSON · Edit on GitHub