OpenCompass's ClinicBench task runs only a 213-item pharmacology-QA slice of a much larger, unrelated-looking 17-dataset clinical benchmark suite published under the same name.
unassessed
| Category | domain |
|---|---|
| Subcategory | pharmacology multiple-choice QA for emerging drugs (a single task from a larger 17-dataset clinical suite) |
| Page status | unknown |
| Metric | accuracy (LLM-judged consistency with the gold letter) |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 213 |
The name "ClinicBench" names two different things, and the OpenCompass task this page documents runs only the smaller of them. OpenCompass's `ClinicBench` dataset config loads a Hugging Face dataset, xuxuxuxuxu/Pharmacology-QA: 213 four-option multiple-choice pharmacology questions, with no citation, licence or description attached to the mirror itself. Matching its exact row count and task description against the wider literature traces this data to one specific task, "Pharmacology QA for Emerging Drugs," inside a much larger benchmark suite also called ClinicBench, published by Fenglin Liu and co-authors (University of Oxford and Amazon) at EMNLP 2024. That original ClinicBench spans three scenarios, eleven tasks and seventeen datasets (over 20,000 test samples in total) covering clinical language generation, understanding and reasoning, evaluated across twenty-two LLMs -- of which OpenCompass implements only this one 213-item pharmacology task, without stating that connection anywhere in its own code.
Four-option multiple-choice pharmacology question (options labelled A-D in the source data); the model is asked to think step by step and give a final "ANSWER: $LETTER" line. The prompt template's own instruction text mentions options "one of ABCDEFGHIJKLMNOP", but that is boilerplate shared across several OpenCompass multiple-choice tasks, not evidence this dataset has more than four options.
No model card in ModelSpec reports this benchmark yet.