HELM's yes/no wrap of Stanford EHRSHOT: predict a future clinical event from EHR codes, scored by exact match, not the paper's AUROC.
unassessed
| Category | domain |
|---|---|
| Subcategory | yes/no future-event prediction from structured EHR code sequences |
| Page status | active |
| Metric | exact_match (HELM); paper reports AUROC and AUPRC |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 6739 |
| Dataset licence | Stanford University Dataset Research Use Agreement |
| Publisher | Stanford University (Shah Lab / CRFM MedHELM packaging) |
This id is HELM's ehrshot scenario, not the original few-shot AUROC benchmark by itself. Each instance is a patient's prior structured EHR codes plus a yes/no question: new diagnosis in a year, abnormal lab if drawn now, long stay, 30-day readmission, or ICU transfer. HELM converts timelines with codes_only and asks the model to answer A/B (yes/no). Inputs are coded events, not clinical notes. English prompts. It is not [ehr_sql](ehr_sql.md).
Zero-shot joint multiple choice. HELM adapter: "Answer A for yes, B for no", max_train_instances 0, max_tokens 1. Scenario references are the strings yes and no. Lab tasks subsample 10,000 labels. Default max_length 100,000 tokens; longer prompts are dropped.
No model card in ModelSpec reports this benchmark yet.