MedHELM three-way classification of whether a clinical note supports, denies, or is uncertain about a patient's history of one of 13 conditions.
unassessed
| Category | domain |
|---|---|
| Subcategory | three-way history classification from English clinical notes (MedHELM) |
| Page status | active |
| Metric | exact_match |
| Direction | higher_is_better |
| Dataset licence | CC BY 4.0 (paper); clinical notes are not a public dump |
| Publisher | Stanford University School of Medicine; MedHELM / Stanford CRFM wrap |
clear is HELM's MedHELM scenario that asks a model to read an English clinical note and decide whether the patient has a history of one named condition. Each condition is a separate run. The thirteen conditions are alcohol dependence, ADHD, bipolar disorder, chronic pain, homelessness, liver disease, major depression, personality disorder, PTSD, substance use disorder, suicidal behavior, tobacco dependence, and unemployment. The paper HELM cites is a retrieval method (CLinical Entity Augmented Retrieval) evaluated on labeled notes; HELM does not run that RAG pipeline. It scores a three-way letter choice on the notes themselves.
Zero-shot joint multiple choice. HELM instructions name A/B/C for has history, does not, or uncertain. max_train_instances=0 and max_tokens=1 in the run spec. The scenario also embeds the same three options in the input. Private Stanford MedHELM entries cap evaluation at 100 instances per condition and pass a local data_path of per-condition .xlsx files.
No model card in ModelSpec reports this benchmark yet.