MedQA

Four-option USMLE-style clinical multiple-choice questions, the most widely reported medical exam benchmark for LLMs.

Also known as: MedQA-USMLE, MedQA-USMLE-4-options

unassessed

What it measures

MedQA tests whether a model can pick the correct answer to a clinical multiple-choice question written in the style of the United States Medical Licensing Examination. Most questions present a short patient vignette (age, presenting symptoms, exam findings, sometimes lab values) and ask for a diagnosis, a next step in management, or an underlying mechanism, then offer several candidate answers. It is a single-turn, English-language, text-only task built from real practice-exam question banks rather than written for the benchmark, so it leans on applied clinical reasoning more than isolated fact recall.

Task format

Four-option multiple-choice clinical vignette question; the model returns a single letter answer (A-D), usually zero-shot or few-shot.

Models reporting this benchmark

Data

This page as JSON · Edit on GitHub