The text-only split of MedXpertQA, ten-option expert-level clinical questions across 17 specialties, which is what OpenCompass registers as MedXpertQA.
unassessed
| Category | domain |
|---|---|
| Subcategory | expert-level clinical multiple-choice |
| Page status | active |
| Metric | accuracy |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 2455 |
| Dataset licence | MIT |
| Publisher | Tsinghua University |
This page covers MedXpertQA Text, the text-only half of MedXpertQA. Each item is an expert-level clinical multiple-choice question with ten lettered options, drawn from USMLE, COMLEX and 17 American specialty-board sources, then filtered, rewritten and expanded so that leaked exam wording is harder to match. Questions are tagged Diagnosis, Treatment or Basic Medicine, and as Reasoning or Understanding. English text only; no images. The multimodal half is a separate page, [medxpertqa_multimodal](medxpertqa_multimodal.md). OpenCompass's `MedXpertQA` config loads this Text test split, not MM.
Ten-option multiple choice. The model returns a single letter A-J. OpenCompass and the authors' reference code use zero-shot generation with a "Among {start} through {end}, the answer is" prompt, then parse the letter.
No model card in ModelSpec reports this benchmark yet.